4 ms·

Having toolchains installed system-wide in the traditional UNIX way is painful, and dare I say it, not the smartest of approaches. If it works for you: great,

by bborud 2y ago

Having toolchains installed system-wide in the traditional UNIX way is painful, and dare I say it, not the smartest of approaches. If it works for you: great, but if you work on a project with multiple developers, sometimes working on multiple projects that have different targets, you will spend a lot of time trying to figure out build and configuration problems. It also doesn't help that people keep using Python for tooling. Why would you want to insist on using a language that brings its own versioning problems and which will behave differently on each developer's computer? I've done embedded development for about a decade now (both as a hobby and professionally) and it is puzzling that people think it is okay to spend a week trying to make everyone's setup do the same thing on a project and then not see how this is a problem. It is a problem. It is annoying. It does waste time. It is unnecessary. Tools should be statically linked binaries. I don't care what language people use to write tools, be it Rust, Go, C, C++. I just wish people would stop prioritizing ad-hoc development and start to make robust tools that can be trusted to work the same way regardless of what's installed on your computer. Python doesn't do that. And it doesn't help that people get angry and defensive instead of taking this a bit more seriously. That being said, things like PlatformIO are a step in the right direction. I know it is a Python project (and occasionally that turns out to be a problem, but less often that for other tools), but they have the right idea: toolchains have to be managed, SDKs have to be managed, libraries have to be managed, project configuration has to be simple, builds have to be reproducible and they have to be reproducible anywhere and anytime. I wish more of the embedded industry would realize that it would be better to have some common efforts to structure things and not always invent their own stuff. I know a lot of people who work for some of the major MCU manufacturers and I am always disheartened when I talk to them because they tend to be very near-sighted: they are mostly busy trying to solve their own immediate problems and tend to not have a great deal of focus on developers' needs.

anymouse123456 2y ago
Just want to second this entire post. It feels like I wrote it myself.
LtdJorge 2y ago
Wouldn't Docker (or other container tools) be a good candidate? Instead of installing the toolchains locally, do the build on a container based on an image with the exact toolchain you need.
- bborud 2y ago
  Yes, I used to do that for a couple of projects, but it wasn't as smooth as I had hoped. For instance I had a bunch of challenges getting USB stuff to work properly. And there were some tools that still required me to lug around a windows laptop for parts of the process. I'd much rather have proper tooling to begin with rather than have to pull out everyone's favorite roll of duct tape (Docker) to hide the mess.
- amluto 2y ago
  It is indeed possible to make a nice container that contains a program, offers a clean interface to that program, and doesn’t package a whole pile of largely irrelevant, guaranteed-to-be-quickly-outdated crap with it. But being possible doesn’t mean that anyone does it. Most containers I’ve ever seen are big messes. Possibly GPL-violating messes if the payload isn’t appropriately licensed. Certainly maintenance and provenance disasters. Unquestionably not reproducibly buildable and possibly not buildable at all.
  - yjftsjthsd-h 2y ago
    > and doesn’t package a whole pile of largely irrelevant, guaranteed-to-be-quickly-outdated crap with it. > Possibly GPL-violating messes if the payload isn’t appropriately licensed. Certainly maintenance and provenance disasters. Unquestionably not reproducibly buildable and possibly not buildable at all. If static binaries are on the table then none of these things can possibly be disqualifiers.
- buescher 2y ago
  For professional embedded development, where reproducible-in-the-future builds are frequently valued, it's common, when possible, to set up your whole toolchain in a vm that's kept under version control.
bschwindHN 2y ago
I refuse to use anything with python tooling now. I just don't. It is just so extremely rare that things work on the first try with it. I have a keyboard project that runs on an RP2040, and the firmware is in Rust. Here are the steps to flash it if you're starting with just the repo and no Rust toolchain: (install rustup) $ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh $ rustup target add thumbv6m-none-eabi $ cargo install elf2uf2-rs * Put the keyboard in bootloader mode * $ cargo run --release This is relying on rustup and Cargo to do the heavy lifting of toolchain management and building, which they are fantastic at. Python projects don't even come close.
- bborud 2y ago
  That looks quite lovely. I really hope that MCU manufacturers invest in Rust and that they both manage to do so in mechanical sympathy with the Rust community and that they realize they have to cooperate on common tooling.
- franga2000 2y ago
  You're comparing the "traditional Linux approach" to curl-pipe-bash, not comparing the actual environments/toolchains. Platformio: curl https://.../get-platformio.py https://.../get-platformio.py | python3 && pio run Pipx: apt install pipx && pipx [build-script.py] Pipenv: pip install --user pipenv pyenv && pipenv run [build-script.py]
  - bborud 2y ago
    They key isn't so much how you install the tools, since that's hopefully something you only do very occasionally, but the quality of life those tools provide once installed.
  - bschwindHN 2y ago
    As mentioned already, it's not about the method of installation. In my experience python-based tools are slower and buggier than their native counterparts, and that's only after going through the installation process. Tools written in C / C++ / Go / Rust just don't have nearly as much fuss, and they run faster and are smaller too.
naitgacem 2y ago
Working with Android app developement has got me spoiled in this area. One needs to just run a command and everything is set up and works the exact same way whether be it locally or on CI with the same version of both build tools and the libraries. It still pains me when I go back to dealing with CMake and Make and trying to get libraries installed, as opposed to say, compile 'library-name-here'.
Gibbon1 2y ago
I have a command line tool I wrote in C# back in 2011 and it just still works.
tpudlik 2y ago
I don't think people choose Python because it's virtues as a language for tooling (it's not great from that perspective, statically linked binaries would be better). They choose it because of the ecosystem of libraries. For data analysis, visualization, and scientific computing, there are no comparable offerings in any other language. And a lot of this is stuff you _really_ don't want to be reimplementing from scratch, because it's very easy to introduce bugs (around numerical stability, say) that cause your software to produce correct results 99% of the time, and plausibly-looking but totally incorrect ones 1% of the time.

by bborud 2y ago

anymouse123456 2y ago

Just want to second this entire post. It feels like I wrote it myself.

LtdJorge 2y ago

Wouldn't Docker (or other container tools) be a good candidate? Instead of installing the toolchains locally, do the build on a container based on an image with the exact toolchain you need.

bborud 2y ago
Yes, I used to do that for a couple of projects, but it wasn't as smooth as I had hoped. For instance I had a bunch of challenges getting USB stuff to work properly. And there were some tools that still required me to lug around a windows laptop for parts of the process. I'd much rather have proper tooling to begin with rather than have to pull out everyone's favorite roll of duct tape (Docker) to hide the mess.
amluto 2y ago
It is indeed possible to make a nice container that contains a program, offers a clean interface to that program, and doesn’t package a whole pile of largely irrelevant, guaranteed-to-be-quickly-outdated crap with it. But being possible doesn’t mean that anyone does it. Most containers I’ve ever seen are big messes. Possibly GPL-violating messes if the payload isn’t appropriately licensed. Certainly maintenance and provenance disasters. Unquestionably not reproducibly buildable and possibly not buildable at all.
- yjftsjthsd-h 2y ago
  > and doesn’t package a whole pile of largely irrelevant, guaranteed-to-be-quickly-outdated crap with it. > Possibly GPL-violating messes if the payload isn’t appropriately licensed. Certainly maintenance and provenance disasters. Unquestionably not reproducibly buildable and possibly not buildable at all. If static binaries are on the table then none of these things can possibly be disqualifiers.
buescher 2y ago
For professional embedded development, where reproducible-in-the-future builds are frequently valued, it's common, when possible, to set up your whole toolchain in a vm that's kept under version control.

bschwindHN 2y ago

I refuse to use anything with python tooling now. I just don't. It is just so extremely rare that things work on the first try with it. I have a keyboard project that runs on an RP2040, and the firmware is in Rust. Here are the steps to flash it if you're starting with just the repo and no Rust toolchain: (install rustup) $ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh $ rustup target add thumbv6m-none-eabi $ cargo install elf2uf2-rs * Put the keyboard in bootloader mode * $ cargo run --release This is relying on rustup and Cargo to do the heavy lifting of toolchain management and building, which they are fantastic at. Python projects don't even come close.

bborud 2y ago
That looks quite lovely. I really hope that MCU manufacturers invest in Rust and that they both manage to do so in mechanical sympathy with the Rust community and that they realize they have to cooperate on common tooling.
franga2000 2y ago
You're comparing the "traditional Linux approach" to curl-pipe-bash, not comparing the actual environments/toolchains. Platformio: curl https://.../get-platformio.py https://.../get-platformio.py | python3 && pio run Pipx: apt install pipx && pipx [build-script.py] Pipenv: pip install --user pipenv pyenv && pipenv run [build-script.py]
- bborud 2y ago
  They key isn't so much how you install the tools, since that's hopefully something you only do very occasionally, but the quality of life those tools provide once installed.
- bschwindHN 2y ago
  As mentioned already, it's not about the method of installation. In my experience python-based tools are slower and buggier than their native counterparts, and that's only after going through the installation process. Tools written in C / C++ / Go / Rust just don't have nearly as much fuss, and they run faster and are smaller too.

naitgacem 2y ago

Working with Android app developement has got me spoiled in this area. One needs to just run a command and everything is set up and works the exact same way whether be it locally or on CI with the same version of both build tools and the libraries. It still pains me when I go back to dealing with CMake and Make and trying to get libraries installed, as opposed to say, compile 'library-name-here'.

Gibbon1 2y ago

I have a command line tool I wrote in C# back in 2011 and it just still works.

tpudlik 2y ago

I don't think people choose Python because it's virtues as a language for tooling (it's not great from that perspective, statically linked binaries would be better). They choose it because of the ecosystem of libraries. For data analysis, visualization, and scientific computing, there are no comparable offerings in any other language. And a lot of this is stuff you _really_ don't want to be reimplementing from scratch, because it's very easy to introduce bugs (around numerical stability, say) that cause your software to produce correct results 99% of the time, and plausibly-looking but totally incorrect ones 1% of the time.