7 ms·
ESP32-S3 has a few SIMD instructions
- adolph 2y agoThe Xtensa processor comes from Cadence and for some reason they like to keep everything under NDA, even information which would help people use their processors. I find it hard to understand why the instruction set should be kept secret; a CPU vendor should make it as easy as possible for engineers to use their CPUs. The P4 can't come soon enough to get off Xtensa.
- jsheard 2y agoThe P4 doesn't have a built-in radio though, so if you want those beefy RISC-V cores you will need to integrate a second ESP32 just to handle WiFi/BT :( It will have USB-OTG and an LCD driver at least, which so far have been missing from all of their RISC-V parts.
- ComputerGuru 2y agoCost optimization aside, that has always been the best way to use an ESP chip. Just go with one of their barebone models wired up as a peripheral to an ARM or RISC mcu.
- clbrmbr 2y agoYet it’s possible to build some incredible applications on top of just ESP32, especially with extra RAM.
- devmunchies 2y agoWe use esp32-s3 at my company (smart speaker) but we don't don anything fancy. Can you explain this? Why use esp as a peripheral if you already have an ARM chip? We were considering moving off of esp to something that would make it easier do cpu-bound AI inference on-device or to enable more advanced audio DSP algos.
- throwup238 2y agoBased on cost and development time, it’s usually just easier to add an ESP and communicate to it using a generic SPI library or something than to add a radio to your PCB and get vendor libraries working on an arbitrary platform.
- timschmidt 2y agoSeems likely they'll continue releasing more models, further integrating the features of the P4 and C6 for example. Maybe we'll even get some risc-v SIMD instructions and support for off-chip SRAM.
- antoniuschan99 2y agoI think that’s totally fine. Might actually be the future direction. But yea would’ve been nice to have wifi/bt integrated. Eg if you want 5ghz then use c5, or if you want some wifi-6 so c6, etc. Also here’s a talk by them on how to use esp as a wifi coprocessor https://youtu.be/g14aEjnjRLw?si=TgkEyJJ2_L_Shuom https://youtu.be/g14aEjnjRLw?si=TgkEyJJ2_L_Shuom There’s also adafruit airlift https://www.adafruit.com/product/4201 https://www.adafruit.com/product/4201
- Aurornis 2y ago> I think that’s totally fine. Might actually be the future direction. Using a second board just for WiFi is definitely not totally fine for most applications. Having everything integrated into a single package important for everything from reducing BOM count to lower power consumption to development simplicity.
- yau8edq12i 2y agoWithout the builtin radio it's really hard to justify the use of an ESP32 over, say, an STM32. The integrated small package with "everything" to make a fun project is the whole appeal.
- AlotOfReading 2y agoEspressif has a huge advantage in lead times over ST recently. I migrated a few projects over because ST couldn't or wouldn't give us supply in under a month when you could buy ESP chips and have them on your doorstep practically overnight.
- vbezhenar 2y agoDid you look at chinese STM clones? We used gd32, I liked it.
- 6SixTy 2y agoWide product range. ARM and RISC-V all called GD32 with an extra letter for the exact line.
- makapuf 2y agoDepends if you want to condone IP theft (compatible independent developments is of course different but I'm not sure this is the case). R&D, Support, good documentation in English and accuracy of specs come at a price.
- pantalaimon 2y agoIs implementing the same peripheral register API really IP theft?
- makapuf 2y agoNo, this was my remark about being compatible.
- sitkack 2y agoThe ESP8684H2 is 1.20 qty 1, more than enough to handle BT an Wifi, then you can use any MCU you want as your application processor.
- ajross 2y agoFWIW this bit in the article is a little confused. The SIMD instructions being detailed appear to be Espressif-custom things implemented using Cadence's "TIE" facility. Cadence does indeed have their own SIMD architecture ("HiFi", really it's a family of similar but binary-incompatible ISAs). And indeed docs for that don't appear in public (though if you look carefully, details for how to emit the instructions are part of the GNU toolchain integration). But that isn't this. If you want docs for this, talk to Espressif, not Cadence.
- mianos 2y agoIt is possible there is some licensing issue around the SIMD, after all it is an optional component. It was available for the LX6 as well, but not included. It's been a good run but it's great the are going to the RISCV, at the very least for the vibe. I have used both architectures, more recently using their esp-idf and it is surprisingly uneventful to switch between them. The only issue I had is the different high/low speed timer devices between chips. In fact it is a surprise the on chip peripheral hardware is incredibly compatible with their idf. Sure, they have a layer for some calls but a lot is just issuing commands to io devices directly, and the same between riskv and tensilkica cores.
- ajb 2y agoXtensa is an unusual beast because its USP (at least, back when it was owned by tensilica) was that you could easily add extensions. Not just off the shelf ones - ones you defined yourself. They had some automation that would generate a toolchain for you to use with your shiny new instructions. Most CPU architectures exist to allow programs written on once implementation to work on another, with Xtensa it's kind of the opposite - it exists to allow each chip to have its own special sauce. Honestly I was a bit surprised that espressif used it without defining their own extension of some kind, if you're not doing that then you might as well use something better known. Edit: ajross* points out that this SIMD extension is such a one, not an off the shelf one. So I guess that explains it. * https://news.ycombinator.com/item?id=40267977 https://news.ycombinator.com/item?id=40267977
- lunfard000 2y agoWas it a secret? You could have guessed that something advertised [0] for "AI" had some kind of SIMD. Even ChatGPT 3.5 can give relevant code to use "AI" features [1]. 0: https://www.espressif.com/en/products/socs/esp32-s3 https://www.espressif.com/en/products/socs/esp32-s3 1: https://chat.openai.com/share/3e1f990d-e8eb-4e56-acbb-ad5a339915fa https://chat.openai.com/share/3e1f990d-e8eb-4e56-acbb-ad5a33...
- iamflimflam1 2y agoNot a secret - just not documented very well if at all. We all knew there were SIMD instructions, but if there’s no information on how to use them or what they do…
- lunfard000 2y agoAnd the author is not documenting them either, just announcing his new niche library. It is not like disassembling a few functions to prove that they exist is dark magic. I just don't see any value in the article.
- iamflimflam1 2y agoI’m not sure I’d call a JPEG decoding library “niche”. There are some numbers here on the performance improvements he’s managed to make. https://atomic14.substack.com/p/even-faster-jpeg-decoding https://atomic14.substack.com/p/even-faster-jpeg-decoding
- lunfard000 2y agomaybe I am missing something but isn't it barely faster than the offical ESP32_JPG? But fair enough, didn't know than JPEG decoding on MCUs is a widespread thing.
- iamflimflam1 2y agoThe "official" version used in that blog post decodes the JPG all in one go - so it's pretty memory hungry. With JPEG encoders that decode sections of the image at a time you can minimise the amount of RAM that needs to be allocated. It's also possible to stream the display data out to screen using DMA while the next chunk of image data is being decoded. It's explained in a bit more details in this original blog post written before the library was optimised: https://atomic14.substack.com/p/the-fastest-esp32-jpeg-decoder https://atomic14.substack.com/p/the-fastest-esp32-jpeg-decod.... It's very easy to forget what a range of MCUs there are, from very puny, to very capable. For example the Espressif range of MCUs - which you'll find in all sorts of consumer products - are very powerful. Couple that with a lot of cheap SPI based display modules and you very quickly start wanting to show images.
- tzmlab 2y agoThere's also a follow-up blog post "ESP32-S3 SIMD Minimal Example" [0]. 0: https://bitbanksoftware.blogspot.com/2024/01/esp32-s3-simd-minimal-example.html https://bitbanksoftware.blogspot.com/2024/01/esp32-s3-simd-m...
- londons_explore 2y agoESP_Sprite, former opensource-projects-guy, now Espressif employee, is the best source of knowledge on this stuff. Looks like back in 2021 they had an intention to document these, but never quite got round to it: https://esp32.com/viewtopic.php?p=88114&sid=f7f25776d9cfc6b611205e20ccf6b9d8#p88114 https://esp32.com/viewtopic.php?p=88114&sid=f7f25776d9cfc6b6... They do publish a bunch of opensource code that uses the SIMD stuff, and an assembler, so it isn't secret, just very badly documented.
- londons_explore 2y agoUpon further inspection, it now seems like it is much better documented... Page 37-301 of the reference manual seems to have all you'd need, including binary instruction encodings, details on instruction timings, etc. https://www.espressif.com/sites/default/files/documentation/esp32-s3_technical_reference_manual_en.pdf https://www.espressif.com/sites/default/files/documentation/...
- jononor 2y agoThis looks pretty decent! And then esp-dsp can be used as example code in some cases. Personally I want to have accelerated Neural Networks for ESP32-S3 using this. Ideally for https://github.com/sipeed/TinyMaix https://github.com/sipeed/TinyMaix which is one of the smallest and most hackable CNN implementations.
- londons_explore 2y agoI had plans to use this SIMD support for some DSP algorithms on camera video feeds.... But looking at how badly documented it is, I may reconsider... Without scatter/gather I don't think I'm gonna be able to meet my timing requirements (I need to distort images through warping, which is tricky to do without scatter/gather)
- amelius 2y agoWhere is a good overview of the various ESP32 chips available and their features?
- mort96 2y agoEspressif has a pretty decent overview on their website: https://www.espressif.com/en/products/socs https://www.espressif.com/en/products/socs They also make a set of modules per chip, so you can get a particular chip in an easier to use package with e.g a built-in PCB antenna or antenna mounting ports or no antenna, various onboard flash sizes, that sort of stuff: https://www.espressif.com/en/products/modules https://www.espressif.com/en/products/modules
- the__alchemist 2y agoS3 if you want more pins and fast C3 if you don't, and are OK with RISC-V PICO 3v2 otherwise.
- tbyehl 2y agoS3 also has USB support that I've come to hugely appreciate on dev boards... tho I just got some oddball single-port boards that used a CH340 anyways. Grrr.
- 15155 2y agoS3 does not have real USB support (in the same way that any STM32 with "USB support" does) - it has a USB-UART/JTAG device that you cannot redefine built-in. (edit: apparently it might? I can't find the documents on how to actually use the OTG device over the fixed IP)
- deleted 2y ago[deleted]
- bitbank 2y agoI think you're thinking of the ESP32-C3. The S3 does have a fully programmable USB port that can do things like HID, mass storage, etc.
- hrydgard 2y agoNice! Could save a couple of cycles per iteration by preloading the shift amounts into several GPRs before entering the loop, instead of initializing them just before use.
- DeathArrow 2y agoWhat about ESP32-C3, using RISC-V architecture, does it also have SIMD instructions?
- guntars 2y agoIt does not. There are RISC-V chips out there (T-Head 906) that have a pre-1.0 vector extension, but these are 64-bit application processors. I'm sure we'll see ESP32 RISC-V chips with SIMD in the next few years.
- fennecfoxy 2y agoYeah I did see that the S3 was meant to have SIMD but with basically no developer support for it (I guess till now?) I was looking at doing some hardware image processing for a project (which I still haven't properly started) and looked at an S3 (too weak even with SIMD) to a Gowin FPGA (cheaper than the competition but FPGAs still seemed like a time sink learning HDL or VHDL) and then I ended up picking a pi zero 2, of which the Videocore IV has great SIMD which is now fully documented. Only problem is that I _still_ haven't started the project even though I have parts sitting there. Was definitely tough trying to consider what the best option might be for processing high speed (90-120fps) video frames.