Viper doesn't do boolean logic?

[Scroungre]

When trying to optimize with micropython.viper it doesn't seem to like doing elementary boolean logic. Am I doing something wrong? Does viper insist that every boolean be a 32-bit int? I dunno. Here's some stripped-back test code:

Code: Select all

#@micropython.viper
def flag_test():
    if y == True:
        x = True
    print(x,y)

x = False
y = True
print(x,y)
flag_test()

With @micropython.viper commented out, it works exactly as expected. When put back in, I get the error:

Code: Select all

Traceback (most recent call last):
  File "<stdin>", line 3, in flag_test
ViperTypeError: can't do binary op between 'object' and 'bool'

I sorta thought that comparing booleans was about the most elementary logic possible.

PS - System is a Pico W, Thonny 4.1.2, Python 3.8.10, running on Windows 7 (x64). Maybe I should just rewrite the whole thing in C instead of trying to optimize Python. *sigh*

[Scroungre]

Hm. Perhaps it's just complaining about uninitialized variable types. This works:

Code: Select all

x = False
y = True

@micropython.native
def flag_test():
    if y == True:
        x = True
    print(x,y)

print(x,y)
flag_test()

But something else is still badly wrong. In my main codebase I'm still getting a different error, namely:

Code: Select all

ViperTypeError: can't do binary op between 'bool' and 'bool'

I will have to write some more 'minimal test code' that actually shows the error.

[ame]

Premature optimisation is the root of all evil.

[Scroungre]

Premature optimisation is the root of all evil.

Except it's NOT premature. The whole thing works fine without optimization, and works even better with micropython.native, but it's still far too slow for the specification. To be honest, I'm still not sure that even with .viper behaving itself it'll be fast enough.

It's only trying to use .viper as the optimizer that's causing all these headaches.

[ame]

Premature optimisation is the root of all evil.

Except it's NOT premature. The whole thing works fine without optimization, and works even better with micropython.native, but it's still far too slow for the specification. To be honest, I'm still not sure that even with .viper behaving itself it'll be fast enough.

It's only trying to use .viper as the optimizer that's causing all these headaches.

Rewrite it in C.

[Scroungre]

Rewrite it in C.

Probably the best bet.

Or get a faster processor (a 'Propeller P2' springs to mind, but their code is not exactly portable if Parallax goes 'pop' and stops making them). Pity - I would have thought an RP2040 at 125MHz plenty fast enough (and no, overclocking it a bit isn't going to give me the speed I need). Oh well... have fun, y'all.

[ame]

There are some optimisation tips here:
https://docs.micropython.org/en/v1.9.3/ ... ython.html

This part may be relevant to your issue:

The argument to a bool cast must be integral type (boolean or integer); when used as a return type the viper function will return True or False objects.

Hopefully someone else will come up with more suggestions.

[hippy]

I will have to write some more 'minimal test code' that actually shows the error.

This seems to be the minimum required to demonstrates the "ViperTypeError: can't do binary op between 'bool' and 'bool'" error -

Code: Select all

@micropython.viper
def test():
  True | True

test()

There do seem to be a few issues with using the 'viper' generator. I would guess the popularity of the RP2040, Pico and Pico W means more people than ever are using MicroPython and that is revealing issues and bugs which haven't been previously identified.

Best thing to do is report them as issues or raise them for discussion, hope the MicroPython Team can fix or resolve those issues or provide workarounds -

https://github.com/micropython/micropython

[jamesh]

Premature optimisation is the root of all evil.

I have never agreed with that. Although it does depend on the definition of premature, which is the biggest get-out clause ever.

[scruss]

I would have thought an RP2040 at 125MHz plenty fast enough (and no, overclocking it a bit isn't going to give me the speed I need). Oh well... have fun, y'all.

Definitely ask on micropython · Discussions · GitHub. Viper is still under-documented, and it still contains obsolete limitation statements like "Functions may have up to four arguments" which were removed over five years ago.

I know you're showing test code, but Viper will be no help at speeding up code containing print() calls.

[hippy]

This was an interesting comment -

Viper code is not always the best choice, since not all of the Python language is supported by it. On the other hand, the byte code and native emitters support all of Python.

That statement is rather dated ( from 2013 ) and early in the life and evolution of MicroPython so it may be outdated now, but it may be that booleans are simply a part of Python which aren't well supported here. That could be by design, an implementation limitation, or oversight.

That 'if y == True:', though valid, isn't something one would normally expect to see, and 'if y:' after 'y = True', plus 'if x == y:' and 'if x != y:' where both are boolean, do as expected. It may be it's a construct that simply wasn't anticipated, or wasn't believed necessary to support. That 'True | True' and similar is a contrivance unlikely to be seen in real code. Things I tried with booleans which weren't contrivances do seem to work as expected though I didn't exhaustively test everything.

The main issue seems to be when using the 'True' and 'False' constants other than for assignments. They appear to remain as 'bool' types rather than as 'object' types, as a variable assigned from 'bool' seems to become. Operations on 'object' and 'bool', plus 'bool' and 'bool', don't appear to be supported. That could perhaps be resolved by internally converting or promoting 'bool' to 'object' which it seems would then be supported.

That may be something which deserves fixing but it looks like viper code will generally be just fine when using booleans, providing one doesn't stray beyond what would normally be done in Python code.

Only the MicroPython team will likely have the insight and knowledge to give a definitive answer.

[PiGraham]

Is there anything relevant in here?
Same error. Global and local variables.

https://github.com/micropython/micropython/issues/8086

[Scroungre]

Different fora do different things for multiple source quotes, so bear with me a little...

First place I found was the optimizing python code page. Their recommendations were implemented before I tried .viper. My objects (as far as I know!) do not change size.

I know, trying to 'print()' is not going to optimize worth a damn. For the speed testing I was doing, there was one:

Code: Select all

start = time.ticks_ms()
... and a lot of other stuff ...
print(time.ticks_diff(time.ticks_ms(), start))

Which I think did what it was supposed to do, but gave me a result I really didn't like. A stopwatch (we're talking at least thirty seconds here) gave roughly the same results.

There may be issues yet with local v. global variables. Experiments along those lines have not been successful. Sticking the entire code in one "def Main():' might think dealt with that, but it didn't. There are no nested definitions.

"If y == True" may not be perfectly "Pythonic", but I like it for readability - and I thought it was logically equivalent to "if y:" I might try that and see if .viper likes it any better.

And I sorta hoped that the MicroPython development team reads this forum occasionally.

Incidentally, @micropython.native does speed up the code and doesn't throw errors - but it's still not fast enough. And we're talking by rather a lot - shaving a few percent here and there isn't going to help. I'm moving to 'C' in hopes of at least 10x faster. Even that'll be marginal.

And thanks to all of you for your help. 's appreciated.

[PiGraham]

Incidentally, @micropython.native does speed up the code and doesn't throw errors - but it's still not fast enough. And we're talking by rather a lot - shaving a few percent here and there isn't going to help. I'm moving to 'C' in hopes of at least 10x faster. Even that'll be marginal.

What does the real code do, how slow is it and how fast does it need to be?

[Scroungre]

What does the real code do, how slow is it and how fast does it need to be?

What it is supposed to do is real-time system control, and the update rate should be about 100kHz.

The existing system runs at 50kHz (+/- a bit) and is made out of dedicated hardware logic - 16v8 GALs and a couple of 74F283s. I don't want to do the same - I want to go faster. I'd be happy with 80kHz -ish - but slower than 50 is just not acceptable.

The same algorithm implemented in basic Python, and on the Pico (default clock, 125MHz), ran at about 1 kHz. With .native, and other optimizations, I got it to about 1.8kHz. I still haven't gotten .viper to work (although it did seem to like the

Code: Select all

if y:

construct).

I don't think rewriting it in C is going to get me to 50kHz. Let alone faster. I'm off by two orders of magnitude.

FYI, some years ago I rewrote the damn thing in Atmel AVR (now Microchip) assembler, doing quadruple-precision math (they're 8bit gizmos, and I need 32-bit resolution) and with a 20MHz clock got about 20kHz out of the algorithm. In quadruple-precision math. Here I am thinking that a native 32-bit processor would be four times faster... *sigh*

[ame]

What does the real code do, how slow is it and how fast does it need to be?

What it is supposed to do is real-time system control, and the update rate should be about 100kHz.

The existing system runs at 50kHz (+/- a bit) and is made out of dedicated hardware logic - 16v8 GALs and a couple of 74F283 (arithmetic chip)s. I don't want to do the same - I want to go faster. I'd be happy with 80kHz -ish - but slower than 50 is just not acceptable.

The same algorithm implemented in basic Python, and on the Pico (default clock, 125MHz), ran at about 1 kHz. With .native, and other optimizations, I got it to about 1.8kHz. I still haven't gotten .viper to work (although it did seem to like the

Code: Select all

if y:

construct).

I don't think rewriting it in C is going to get me to 50kHz. Let alone faster. I'm off by two orders of magnitude.

FYI, some years ago I rewrote the damn thing in Atmel AVR (now Microchip) assembler, doing quadruple-precision math (they're 8bit gizmos, and I need 32-bit resolution) and with a 20MHz clock got about 20kHz out of the algorithm. In quadruple-precision math. Here I am thinking that a native 32-bit processor would be four times faster... *sigh*

It would be if you weren't running Python.

[Scroungre]

Which is why I'm dragging myself through five hundred yards of [stuff - mod edit, please keep language child friendly] trying to install a C SDK that'll program my pico W's!

I can write C code in Notepad, for crying out loud, but I like having an IDE and an SDK and things like context illumination. Fnord. I like being able to click on a few icons to download and program instead of having to pop out to the command line and whack away at uncached file paths. I like it that directories and their paths are made for me, and that compilation options are found in little tick-boxes instead of obscure command-line options.

It's much better than it was. And I like it that way.

{Edited to note that mod edit is entirely correct. My bad.}

[PiGraham]

I don't know, but 100KHz real-time control in micropython wouldn't be my first choice.

If it's a simple system C SDK is probably the way to go.

It it's complex there are some RTOS options
https://www.bing.com/search?q=realtime+ ... 41016069c6

[hippy]

Incidentally, @micropython.native does speed up the code and doesn't throw errors - but it's still not fast enough. And we're talking by rather a lot - shaving a few percent here and there isn't going to help. I'm moving to 'C' in hopes of at least 10x faster. Even that'll be marginal.

MicroPython is by default compiled to bytecode which is then interpreted. The generally accepted rule of thumb is it will be about 100 times slower than the equivalent code written in C. Using 'native' and 'viper' code generation will move that towards being about twice as fast, perhaps more if the code suits that.

Much will depend on exactly what one is doing and there can still be interpreter overhead in calling the 'native' and 'viper' code and those calling interpreted functions. The more one can put in a 'native' or 'viper' routine, the longer one can stay within those, the greater the gains will usually be.

I did some benchmarking here - viewtopic.php?t=303458 - That was a while ago and both MicroPython and desktop Python have improved over time, but I would expect it's still in the right ballpark. It's an artificial benchmark and most real world code probably won't see such great 'native' and 'viper' benefits.

Python and MicroPython's virtues are in making coding easier, and that come at the cost of being interpreted and being slower to execute. There are Python compilers but they are usually only available for 'desktop platforms'.

If one does need fast execution, Python and MicroPython may not be the solutions and one may need to use C, and may need to optimise that, overclock the RP2040. If the RP2040 isn't fast enough one may have to consider a faster microcontroller or an SBC SoC, using a Pi Zero or a faster Pi variant.

What it is supposed to do is real-time system control, and the update rate should be about 100kHz.

The same algorithm implemented in basic Python, and on the Pico (default clock, 125MHz), ran at about 1 kHz. With .native, and other optimizations, I got it to about 1.8kHz.

100kHz means getting everything done in 10us; that's about 1,250 instructions on an RP2040 at default clock speed.

If you can describe the functionality needed then people here would be better placed to offer advice, determine the feasibility of using an RP2040 or what you would need, whether things like PIO and the interpolator may be helpful in what you are doing.

[hippy]

And I sorta hoped that the MicroPython development team reads this forum occasionally.

Jimmo occasionally does but MicroPython has a small team with limited resources, it's a multi-platform product, not a Raspberry Pi offering or something specific to RP2040. The team and other contributors generally deal with issues and discussions on the MicroPython GitHub.

[jamesh]

Difficult to tell without more information, but I suspect a combination of using C and the PIO would easily make this possible. Python should not be used for real time tasks.

[Scroungre]

Python should not be used for real time tasks.

I think I have learned that, albeit the hard way. Python's nice for making LEDs go blinky.

The remark that C is known to be good for an about 100x speedup is rather heartening. Who knows, it might work - if I can get the SDK working (whee).

The timing is eventually to be run by a regular 'tick' signal, at about 100kHz*, and everything has to be done before the next tick shows up. That's what the other core of the RP2040 was being reserved for (that and user interfacing).

I don't think the PIO is sophisticated enough. There's about eight boolean flags that determine 'add what to where' in the algorithm, and each further operation is dependent on the previous one. I was going to use the PIO for output timing - I need a reasonably precise 1uS long pulse but if I can't crank out whether it should pulse or not at 50kHz, there's not much point in trying.

I'd like to tell you more about it, but there's all kinds of interesting lawyers who wear much more expensive clothing than I do in the way. Like I said, there's an existing system that does what it does - built out of TTL chips** - but I'm trying to do better.

* As I've said, it doesn't have to be exactly 100kHz. But less than 50kHz is not acceptable.
** Pottering along at a 4MHz clock. It's wonderful what you can do by writing your own opcodes.

[hippy]

Like I said, there's an existing system that does what it does - built out of TTL chips** - but I'm trying to do better.
** Pottering along at a 4MHz clock. It's wonderful what you can do by writing your own opcodes.

Maybe FPGA is the direction you need to be heading in ?

This looked interesting and there are perhaps others - https://www.electronics-lab.com/pico-ic ... ice40-fpga

[Scroungre]

Where I went in the first place was CPLDs, specifically the Xilinx Xc9500 family. Then they stopped making them.