10 ms·

Benchmarking MicroPython

22 points by ibobev 1y ago

drewcoo 1y ago
Somehow, I don't mind that the code blew the stack because it was recursion without memoization. Blowing the stack should be a pretty clear sign to figure out why and fix it.
mbirth 1y ago
I'm missing testing the different emitters as demonstrated here: https://www.kickstarter.com/projects/214379695/micro-python-python-for-microcontrollers/posts/664832 https://www.kickstarter.com/projects/214379695/micro-python-... Not sure whether they're supported on all the architectures, though.
- jononor 1y ago
  I have added this now, as a reply to one of the other posts :) 100x speedup.
snvzz 1y ago
>pico 2w ARM or RISC-V?
mrheosuper 1y ago
Generate 2000 random numbers and sorting them using bubble sort on 160Mhz 32bit mcu takes 80 seconds ? This is exactly why micropython is a toy.
- jononor 1y ago
  The same can be done in 0.02 seconds with MicroPython. I posted optimized code here, https://github.com/jonnor/embeddedml/tree/master/handson/micropython-sort https://github.com/jonnor/embeddedml/tree/master/handson/mic...
zem 1y ago
would have been nice to see benchmarks against the equivalent c code running on the same microprocessor, not against micropython code running on different hardware. the post just told me that microprocessors are slow, not how well micropython performs.
deleted 1y ago
[deleted]
wewewedxfgdf 1y ago
As the author points out, performance is pretty much irrelevant - it is a rewrite of python prioritizing memory usage.
jononor 1y ago
This should not be used to conclude on the viability of using MicroPython for numeric type tasks. For that one should at least take into account the following: Integers are much faster than floats (floats involve a pointer and a heap allocation, integers are stored in a single word/object). array.array is preferred over list for something like sort. Continuous memory representation of numbers versus general purpose list of objects. MicroPython has on-board "JIT" (native/viper emitters), have to explicitly annotate the function. Should give 4-10x improvements for this case. MicroPython has an on-board assembler, so one can write ARM assembly and get that to a function. MicroPython also has support for C modules, which expose a Python API. Including dynamic native modules which can be installed at runtime with the package manager. Bubblesort is O(n*2), which hurts for even a few thousand numbers. Actual sorting on a microcontroller should be done with an O(n log n) algorithm.
- jononor 1y ago
  Ok, you guys have successfully nerd sniped me this morning... Here some experiments showing the use of code emitter to speed this code up massively. Link to code: https://github.com/jonnor/embeddedml/tree/master/handson/micropython-sort https://github.com/jonnor/embeddedml/tree/master/handson/mic... The results on ESP32S3 (M5Stick AtomS3U), running MicroPython 1.24.1. All times in seconds, for the 2000 number sort. bubble.py 19.119 bubble_native.py 9.482 bubble_viper.py 0.904 heapsort_viper.py 0.02 So one can do 100x better without changing the algorithm, and 1000x by changing it :) Microcontrollers are a constrained platform. They can be plenty fast - but it is much more important to use the tools available, compared to on PC. MicroPython has excellent tools available for this.
  - jononor 1y ago
    EDIT: 20x better with same algorithm, not 100x. Do not post before the coffee starts kicking in...
- jononor 1y ago
  If anyone is interested in fast dynamic native C modules for MicroPython, you can check out emlearn-micropython - a machine learning and digital signal processing library. https://github.com/emlearn/emlearn-micropython https://github.com/emlearn/emlearn-micropython Disclaimer: I am the maintainer
Archit3ch 1y ago
You can drop down to C and call it from MicroPython if you want to count cycles. Of course, you have to recompile your C every time it changes, which is annoying when you're used to the REPL workflow.

22 points by ibobev 1y ago

Somehow, I don't mind that the code blew the stack because it was recursion without memoization. Blowing the stack should be a pretty clear sign to figure out why and fix it.

mbirth 1y ago

I'm missing testing the different emitters as demonstrated here: https://www.kickstarter.com/projects/214379695/micro-python-python-for-microcontrollers/posts/664832 https://www.kickstarter.com/projects/214379695/micro-python-... Not sure whether they're supported on all the architectures, though.

jononor 1y ago
I have added this now, as a reply to one of the other posts :) 100x speedup.

snvzz 1y ago

>pico 2w ARM or RISC-V?

mrheosuper 1y ago

Generate 2000 random numbers and sorting them using bubble sort on 160Mhz 32bit mcu takes 80 seconds ? This is exactly why micropython is a toy.

jononor 1y ago
The same can be done in 0.02 seconds with MicroPython. I posted optimized code here, https://github.com/jonnor/embeddedml/tree/master/handson/micropython-sort https://github.com/jonnor/embeddedml/tree/master/handson/mic...

zem 1y ago

would have been nice to see benchmarks against the equivalent c code running on the same microprocessor, not against micropython code running on different hardware. the post just told me that microprocessors are slow, not how well micropython performs.

deleted 1y ago

[deleted]

wewewedxfgdf 1y ago

As the author points out, performance is pretty much irrelevant - it is a rewrite of python prioritizing memory usage.

jononor 1y ago

This should not be used to conclude on the viability of using MicroPython for numeric type tasks. For that one should at least take into account the following: Integers are much faster than floats (floats involve a pointer and a heap allocation, integers are stored in a single word/object). array.array is preferred over list for something like sort. Continuous memory representation of numbers versus general purpose list of objects. MicroPython has on-board "JIT" (native/viper emitters), have to explicitly annotate the function. Should give 4-10x improvements for this case. MicroPython has an on-board assembler, so one can write ARM assembly and get that to a function. MicroPython also has support for C modules, which expose a Python API. Including dynamic native modules which can be installed at runtime with the package manager. Bubblesort is O(n*2), which hurts for even a few thousand numbers. Actual sorting on a microcontroller should be done with an O(n log n) algorithm.

jononor 1y ago
Ok, you guys have successfully nerd sniped me this morning... Here some experiments showing the use of code emitter to speed this code up massively. Link to code: https://github.com/jonnor/embeddedml/tree/master/handson/micropython-sort https://github.com/jonnor/embeddedml/tree/master/handson/mic... The results on ESP32S3 (M5Stick AtomS3U), running MicroPython 1.24.1. All times in seconds, for the 2000 number sort. bubble.py 19.119 bubble_native.py 9.482 bubble_viper.py 0.904 heapsort_viper.py 0.02 So one can do 100x better without changing the algorithm, and 1000x by changing it :) Microcontrollers are a constrained platform. They can be plenty fast - but it is much more important to use the tools available, compared to on PC. MicroPython has excellent tools available for this.
- jononor 1y ago
  EDIT: 20x better with same algorithm, not 100x. Do not post before the coffee starts kicking in...
jononor 1y ago
If anyone is interested in fast dynamic native C modules for MicroPython, you can check out emlearn-micropython - a machine learning and digital signal processing library. https://github.com/emlearn/emlearn-micropython https://github.com/emlearn/emlearn-micropython Disclaimer: I am the maintainer

Archit3ch 1y ago

You can drop down to C and call it from MicroPython if you want to count cycles. Of course, you have to recompile your C every time it changes, which is annoying when you're used to the REPL workflow.