User avatar
OneMadGypsy
Posts: 359
Joined: Wed Apr 28, 2021 1:57 am
Location: New Orleans, Louisiana
Contact: Website

Arguments In USER_C_MODULE methods

Thu Sep 23, 2021 4:37 pm

I personally use the below technique for accepting arguments in USER_C_MODULE methods. It openly accepts every form of argument and has no restriction on how an individual argument can be provided.

Code: Select all

STATIC mp_obj_t GFXBuff_rgb565(mp_uint_t n_args, const mp_obj_t *args, mp_map_t *kw_args) {
    enum {ARG_self, ARG_hex};
    static const mp_arg_t allowed_args[] = {
        { MP_QSTR_self  , MP_ARG_REQUIRED | MP_ARG_OBJ, {.u_obj = MP_ROM_NONE}},
        { MP_QSTR_hex   , MP_ARG_REQUIRED | MP_ARG_INT, {.u_int = 0}},
    };
    
    mp_arg_val_t kw[MP_ARRAY_SIZE(allowed_args)];
    mp_arg_parse_all(n_args, args, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, kw);
    
    return mp_obj_new_int(RGB565(kw[ARG_hex].u_int));
}

MP_DEFINE_CONST_FUN_OBJ_KW(GFXBuff_rgb565_obj, 0, GFXBuff_rgb565);

However, tons of USER_C_MODULE examples do not use this method and opt to restrict arguments by arg and kwarg, as-well-as the number of args that can be accepted. I'm struggling to figure out why. IMO, you would want your micropython implementation to resemble, as closely as possible, the features/behavior that python allows for arguments. This leads me to believe that maybe there is some optimization or other such benefit to restricting the argument interface. I would like to understand what those optimizations or benefits (if any) are, but USER_C_MODULES is one of the most poorly documented things I've ever had to work with. There is only so much that I can determine from observation.

I am hoping that someone that truly understands this system will see this post, and be kind enough to explain how the various ways that this can be done affect ... whatever it affects (performance, optimization, etc).
"Focus is a matter of deciding what things you're not going to do." ~ John Carmack

hippy
Posts: 10741
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Arguments In USER_C_MODULE methods

Thu Sep 23, 2021 6:37 pm

OneMadGypsy wrote:
Thu Sep 23, 2021 4:37 pm
However, tons of USER_C_MODULE examples do not use this method and opt to restrict arguments by arg and kwarg, as-well-as the number of args that can be accepted. I'm struggling to figure out why.
My answer would be that it's easier, saves a whole lot of work. Authors may not know how to do it better, not care to, and often don't have the time to look into doing it other than how they do. As you note it's not well documented and there aren't that many hours in a day.

That's my excuse anyway -- It's that "good enough" argument again :D

User avatar
OneMadGypsy
Posts: 359
Joined: Wed Apr 28, 2021 1:57 am
Location: New Orleans, Louisiana
Contact: Website

Re: Arguments In USER_C_MODULE methods

Thu Sep 23, 2021 7:37 pm

I hope that is the actual reason. That would mean that I don't need to potentially reformat thousands of lines of code to squeeze out some (probably tiny) amount of performance. BUT, is it really the answer though? Why would micropython allow you to handle arguments in so many different ways, when all of them except for the way I am doing it aren't pythonic on the front end? Did they just flesh out a bunch of possibilities or is there a damn good reason why these possibilities exist? I like your answer, but only because it makes my life a lot easier. I still want to know (officially) if my life should be hard.

I could totally see all the options being endemic to the design. I mean, no matter what you do it's going to get dumped into this:

Code: Select all

 mp_arg_val_t kw[MP_ARRAY_SIZE(allowed_args)];
 mp_arg_parse_all(n_args, args, kw_args, MP_ARRAY_SIZE(allowed_args), allowed_args, kw);

But which one begat the other? It could just as easily be looked at from the other direction, and if it is, were right back to my original question. In more simple terms, do we parse args because there are so many options or are there so many options because we can parse args? Unfortunately, I think I am going to have to rip apart the entire system to find out, but I need to become more knowledgeable to truly understand whatever ramifications I discover, maybe even to discover ramifications in the first place. Hopefully someone high up in the ranks of our miniature serpent overlord will see this and chime in.

I'll tell ya' ~ give me another year and I'm going to end up working for micropython. That's not my goal, but with as hardcore as I go I'll be a damn good candidate by then. I already tore apart half of the system just to be able to do any of this in the first place, and I don't stop. Sometimes, I'll sleep like 2 hours and be abruptly woken up by curiosity, and then spend the next X hours filling in my mental map of how all of this is constructed. "Can't" isn't even an option. "Can" is just a stepping-stone to "Will". I FULLY believe in and utilize the power of intention, and I'm intent on a lot of things. One of them is MASTERING this. ALL of it.
"Focus is a matter of deciding what things you're not going to do." ~ John Carmack

hippy
Posts: 10741
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 10:46 am

OneMadGypsy wrote:
Thu Sep 23, 2021 7:37 pm
Why would micropython allow you to handle arguments in so many different ways, when all of them except for the way I am doing it aren't pythonic on the front end?
I'll be honest; I'm not clear on what you are doing, what issue you perceive there to be, what isn't Pythonic.

You haven't detailed what the actual problem is, have only declared that it exists, but I presume it comes down to only being able to use things like 'x = mymathlib.add(2, 3)' and not 'mymathlib.add(lhs=2, rhs=3)' or the more obtuse 'mymathlib.add(rhs=3, lhs=2)', and potentially compounded by arguments only allowed to be integer and not float.

I guess that comes about because there isn't a mandated way to do it, so everyone just follows the simplistic way of fixed arguments or copies what has been done elsewhere in code.

For MicroPython supplied code I expect it's that "good enough" again, isn't perceived as a problem, or isn't seen as enough of a problem to put it ahead of other things.
OneMadGypsy wrote:
Thu Sep 23, 2021 7:37 pm
Hopefully someone high up in the ranks of our miniature serpent overlord will see this and chime in.
I have seen 'jimmo' occasionally venture into this forum, who is a member of the MicroPython team, but taking your questions to the MicroPython forum may be the best route to getting them comprehensively or definitively answered - https://forum.micropython.org

I suspect most of it will come down to "that's just how it is, how things evolved" because that's usually the way things come about rather than from long discussions and analysis of how things would best be done.
OneMadGypsy wrote:
Thu Sep 23, 2021 7:37 pm
I FULLY believe in and utilize the power of intention, and I'm intent on a lot of things. One of them is MASTERING this. ALL of it.
I would welcome a more abstract and universal way to easily handle method parameters, and I am sure others would, so I hope you do, can get the MicroPython team on-board to implement change and improve MicroPython. I am sure they would appreciate all the help they can get.

User avatar
OneMadGypsy
Posts: 359
Joined: Wed Apr 28, 2021 1:57 am
Location: New Orleans, Louisiana
Contact: Website

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 2:08 pm

You haven't detailed what the actual problem is,
I'm not sure that there is any problem, just possibilities that don't seem necessary or even congruent with what you would expect to be allowed from the front end. Those possibilities simply spark curiosity as to whether they are some form of an optimization or just noise.
"Focus is a matter of deciding what things you're not going to do." ~ John Carmack

User avatar
OneMadGypsy
Posts: 359
Joined: Wed Apr 28, 2021 1:57 am
Location: New Orleans, Louisiana
Contact: Website

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 3:21 pm

and potentially compounded by arguments only allowed to be integer and not float.

That's not true.

I crammed the below codes into one of my existing methods for example purposes. EDIT: If you don't build from scratch you will get an error that "MP_QSTR_f" is already in use. I changed the below codes to use "fl", but I did not remake this image to reflect that change.
Image

Put this in your module.

Code: Select all

typedef struct _mp_obj_float_t {
    mp_obj_base_t base;
    mp_float_t value;
} mp_obj_float_t;

const mp_obj_float_t def_float = {{&mp_type_float}, 0.0f};

Do this in your args ... making sure you have ARG_fl in your argument enum.

Code: Select all

{ MP_QSTR_fl, MP_ARG_OBJ, {.u_rom_obj = MP_ROM_PTR(&def_float)} },

Use it like this

Code: Select all

float fl = mp_obj_float_get(kw[ARG_fl].u_obj);
mp_printf(MP_PYTHON_PRINTER, "my float is: %f\n",  fl);

Basically we are just cherry-picking parts of objfloat.c. Imagine if your statement was true. That would be crazy ~ an entire fundamental type would be eliminated. I'm assuming I must have misunderstood what you meant. You are quite intelligent, and I find it hard to believe that you would truly believe you can't send floats to the back end. There's nothing that you can't do. Some things are just more cumbersome than others.
"Focus is a matter of deciding what things you're not going to do." ~ John Carmack

hippy
Posts: 10741
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 6:32 pm

OneMadGypsy wrote:
Fri Sep 24, 2021 2:08 pm
You haven't detailed what the actual problem is,
I'm not sure that there is any problem, just possibilities that don't seem necessary or even congruent with what you would expect to be allowed from the front end. Those possibilities simply spark curiosity as to whether they are some form of an optimization or just noise.
Perhaps provide some concrete example which illustrates what the issue is or how it manifests itself and perhaps things will be clearer for me as to how you mean.
OneMadGypsy wrote:
Fri Sep 24, 2021 3:21 pm
and potentially compounded by arguments only allowed to be integer and not float.
That's not true. ...

I'm assuming I must have misunderstood what you meant. You are quite intelligent, and I find it hard to believe that you would truly believe you can't send floats to the back end. There's nothing that you can't do. Some things are just more cumbersome than others.
I think we are talking about different things. I was meaning there's often a problem that the MicroPython user cannot specify floats because the C Module author has only allowed integers to be specified, for example -

Code: Select all

// .add(lhs, rhs)
STATIC mp_obj_t mymathlib_add(mp_obj_t lhs_obj, mp_obj_t rhs_obj) {
    mp_int_t lhs = mp_obj_get_int(lhs_obj);
    mp_int_t rhs = mp_obj_get_int(rhs_obj);
    mp_int_t res = lhs + rhs;
    return mp_obj_new_int(res);
}
MP_DEFINE_CONST_FUN_OBJ_2(mymathlib_add_obj, mymathlib_add);
And that comes about because there is no simple and elegant way of handling variant arguments. As you say floats could be handled, keywords could be handled, it is just as you say "cumbersome", extra work, and most people are adverse to doing more than the least they can get away with.

A good example of that is 'machine.pwm.freq(hz)' where 'hz' can only be an integer not a float even though the Pico can easily generate fractional PWM frequencies.

Worse, in that particular case, it appears it has has been decided that it will be this way, only accepting integers, and there won't be any easy means to make it any other way for ports which could support it without jumping through hoops, creating a whole lot of work and a code maintenance nightmare for that port.

User avatar
OneMadGypsy
Posts: 359
Joined: Wed Apr 28, 2021 1:57 am
Location: New Orleans, Louisiana
Contact: Website

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 10:35 pm

Perhaps provide some concrete example which illustrates what the issue is or how it manifests itself and perhaps things will be clearer for me as to how you mean.

I feel like I've already made it very clear. I will do a better job right now. There are 7 ways to format a function to accept arguments, only one of them is necessary.

None of these are necessary unless there is some underlying optimization that I am not aware of. They do not emulate python behavior. They do not make sense from a python perspective, because this is not how python works. IMO (barring something I may be missing) these are junk methods that serve no purpose but to unnecessarily make things confusing/complicated.

Code: Select all

#define MP_DEFINE_CONST_FUN_OBJ_0(obj_name, fun_name)
#define MP_DEFINE_CONST_FUN_OBJ_1(obj_name, fun_name)
#define MP_DEFINE_CONST_FUN_OBJ_2(obj_name, fun_name)
#define MP_DEFINE_CONST_FUN_OBJ_3(obj_name, fun_name) 
#define MP_DEFINE_CONST_FUN_OBJ_VAR(obj_name, n_args_min, fun_name) 
#define MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(obj_name, n_args_min, n_args_max, fun_name) 

You could use this for everything, and perfectly emulate python behavior on the front end. This is the only version that makes any sense.
#define MP_DEFINE_CONST_FUN_OBJ_KW(obj_name, n_args_min, fun_name)


There is no problem or issue. There is nothing about these that I don't understand other than "Why make 6 extra ways that break expected python behavior?". That's been the question all along ~ Why? If there is no real answer, then the real answer is "For absolutely no good reason." However, like you said, most people don't want to do more than they have to so, there must be some reason why these exist and I want to know it. The only reason I can gather is: If you are willing to break your argument interface, you can type a little less code in the back end. If that is the real reason then these are junk functions, IMO. USER_C_MODULES are already heavy on the boilerplate, this isn't a notable optimization in that regard, and being able to break your front end just to type a little less is a terrible idea.
"Focus is a matter of deciding what things you're not going to do." ~ John Carmack

hippy
Posts: 10741
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Arguments In USER_C_MODULE methods

Fri Sep 24, 2021 11:57 pm

OneMadGypsy wrote:
Fri Sep 24, 2021 10:35 pm
They do not emulate python behavior. They do not make sense from a python perspective, because this is not how python works. ... "Why make 6 extra ways that break expected python behavior?".
I am not seeing how they break Python behaviour. They seem to reflect the way things would be from the MicroPython programmers perspective -

Code: Select all

def f():        #define MP_DEFINE_CONST_FUN_OBJ_0(obj_name, fun_name)
def f(a):       #define MP_DEFINE_CONST_FUN_OBJ_1(obj_name, fun_name)
def f(a, b):    #define MP_DEFINE_CONST_FUN_OBJ_2(obj_name, fun_name)
def f(a, b, c): #define MP_DEFINE_CONST_FUN_OBJ_3(obj_name, fun_name) 
def f(*args):   #define MP_DEFINE_CONST_FUN_OBJ_VAR(obj_name, n_args_min, fun_name) 
def f(*args):   #define MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(obj_name, n_args_min, n_args_max, fun_name)
While every C module could use a single generic definition for each method that would leave it to every method to check it had been passed the required number of arguments. These are just shortcuts to save having to do that.

If there were only a single generic definition you would need something like the following in every method -

Code: Select all

    if (n_args < 1) {
        mp_raise_TypeError(MP_ERROR_TEXT("at least one argument expected"));
    }
    if (n_args > 2) {
        mp_raise_TypeError(MP_ERROR_TEXT("only two arguments allowed"));
    }
Anyone sensible would soon replace that, even if just to save typing and to ensure consistent error text, with something like -

Code: Select all

    check_arg_number(n_args, 1, 2);
The various MP_DEFINE_CONST_FUN_OBJ definitions do that for you, saves even having to do that.

I see it as a convenience and not much different to providing 'inc(x)' and 'dec(x)' functions when one would really only need 'add(x,n)'.

And one has to remember this is MicroPyhon not Python. MicroPython is intended to run on constrained systems where supporting keyword parameters means having to specify them in some data table which eats up space and slows down execution. And in most cases the users won't be using keyword parameters or will accept an author's lack of support for them.

Return to “MicroPython”