4 ms·

The place where I run into Python is mostly while doing embedded development as the tooling for platforms like Zephyr and ESP-IDF. And they are notoriously brit

by bborud 4y ago

The place where I run into Python is mostly while doing embedded development as the tooling for platforms like Zephyr and ESP-IDF. And they are notoriously brittle. Which has a high cost in terms of lost productivity. So we aren't talking about scripts I maintain, but scripts that are part of the tooling for platforms I use. The point is that it doesn't matter what a theoretical developer could do to lower the "cost of ownership" for their end users, but what is actually being done by actual projects. And in the embedded sphere, the situation is pretty sad. You will have to spend a few weeks every year trying to sort out the tooling. And it isn't like you the user are supposed to hack the tooling. That's not part of the workflow. Or that it is vitally important that one can do quick changes to the tooling while working on it. It really isn't. It just has to work. Why should the end user have to care about whether you use pyenv or conda? Or have to figure out how to make them play nicely with some code they didn't write and which they only want to run with a minimum of hassle? It is kind of like demanding you understand and set the timing advance on your car's engine. Why on earth would you even want to know of it, much less fiddle with it?

palotasb 4y ago
I'm not sure about the problem of brittleness caused by buggy scripts you must rely on -- you'd get that in any language. But if the brittleness is caused by slightly different dependencies being installed for each developer or CI machine, I also recommend creating development environment setup and activation scripts that create a reproducible development environment (based on a requirements.txt generated via pip freeze, committed into the repo; or similar). Even if the platforms you depend on don't do this, you can do it yourself. It won't fix their bugs, but it will make the development environment more deterministic. I agree that not all embedded developers should have to work with daily or even monthly. Ideally, all they have to do is run to commands: ./scripts/setup-dev-env source ./scripts/activate-dev-env However, embedded software development heavily relies on invoking build systems, compilers, analyzers and other tools -- calling project-specific programs and scripts. At least one person on each project should be familiar with setting these up for the 1st time, document their usage, and support others.
- bborud 4y ago
  The end user of the software could do all that, yes. Or the tooling could have been done in a manner that didn't make it so brittle the user has to come up with mitigation strategies to keep things running.
bb88 4y ago
As a professional python dev, I understand your pain. It took me a while to sort out the pyenv/virtualenv/conda stuff myself. Could python be better in this regard? Yes, absolutely. I wrote a comment a while back to help out someone else with python packaging. Once you get your mind around it, it's not too bad frankly. https://news.ycombinator.com/item?id=32122191 https://news.ycombinator.com/item?id=32122191
- bborud 4y ago
  The perspective I'm trying to shine a light on is what it is like to be the consumer of tooling written in Python - not the author. I was a bit unclear in the title (which is what most people seem to have read, and then skimmed the blog posting), but the tooling I'm talking about is things like the `west` tool for Zephyr or the `idf.py` tool for ESP-IDF. Tools that are centrally maintained and distributed. Not the ad-hoc tools people use to do various chores in their codebases. Both of the tools mentioned are notorious for being extremely brittle. Actually, as we speak, I'm trying to figure out why a project that worked before I left on vacation, and which has been untouched for several weeks, now suddenly doesn't build (because the tooling doesn't run). This is actually par for the course. You have to expect things to break for no obvious reason. My guess? I probably updated some other piece of software and it upgraded a common dependency. I spend about 10-20% of my time dealing with this sort of nonsense.
  - bccdee 4y ago
    It sounds like your main issue is that you don't isolate your dependencies into virtual environments. That involves jumping through a couple hoops, sure (primarily just invoking `./venv/bin/activate` before you run the script), but for the most part it entirely resolves Python dependency problems. Could Python's dependency management be less clunky? Absolutely. But ultimately, when it comes to the tools you mentioned, the vendor is the one responsible for not explaining proper Python dependency management in their documentation, or for not creating an ergonomic wrapper around it.
  - bb88 4y ago
    If you have issues like that, definitely look into virtualenvs and requirements.txt. It's meant to isolate python environments from breaking like that.
dangerface 4y ago
Tooling for embedded development? Please back in my day you just didn't get any tooling even the programer was left to the manufacture to make. > It is kind of like demanding you understand and set the timing advance on your car's engine. Why on earth would you even want to know of it, much less fiddle with it? Developers are not end users a technical understanding and capability is to be expected you aren't talking about the tooling for a pc you are talking about the tooling for a microcontroller. A more accurate example would be the timing belt manufacture expects the car manufacture to provide their own tooling to put the timing belt on their engine, in the same way a pic manufacture expects you to provide your own tooling. The end user is whoever buys the device you build with the pic in the same way that the end user of a timing belt is the person who buys the car not the manufacture that makes the car.

by bborud 4y ago

palotasb 4y ago

I'm not sure about the problem of brittleness caused by buggy scripts you must rely on -- you'd get that in any language. But if the brittleness is caused by slightly different dependencies being installed for each developer or CI machine, I also recommend creating development environment setup and activation scripts that create a reproducible development environment (based on a requirements.txt generated via pip freeze, committed into the repo; or similar). Even if the platforms you depend on don't do this, you can do it yourself. It won't fix their bugs, but it will make the development environment more deterministic. I agree that not all embedded developers should have to work with daily or even monthly. Ideally, all they have to do is run to commands: ./scripts/setup-dev-env source ./scripts/activate-dev-env However, embedded software development heavily relies on invoking build systems, compilers, analyzers and other tools -- calling project-specific programs and scripts. At least one person on each project should be familiar with setting these up for the 1st time, document their usage, and support others.

bborud 4y ago
The end user of the software could do all that, yes. Or the tooling could have been done in a manner that didn't make it so brittle the user has to come up with mitigation strategies to keep things running.

bb88 4y ago

As a professional python dev, I understand your pain. It took me a while to sort out the pyenv/virtualenv/conda stuff myself. Could python be better in this regard? Yes, absolutely. I wrote a comment a while back to help out someone else with python packaging. Once you get your mind around it, it's not too bad frankly. https://news.ycombinator.com/item?id=32122191 https://news.ycombinator.com/item?id=32122191

bborud 4y ago
The perspective I'm trying to shine a light on is what it is like to be the consumer of tooling written in Python - not the author. I was a bit unclear in the title (which is what most people seem to have read, and then skimmed the blog posting), but the tooling I'm talking about is things like the `west` tool for Zephyr or the `idf.py` tool for ESP-IDF. Tools that are centrally maintained and distributed. Not the ad-hoc tools people use to do various chores in their codebases. Both of the tools mentioned are notorious for being extremely brittle. Actually, as we speak, I'm trying to figure out why a project that worked before I left on vacation, and which has been untouched for several weeks, now suddenly doesn't build (because the tooling doesn't run). This is actually par for the course. You have to expect things to break for no obvious reason. My guess? I probably updated some other piece of software and it upgraded a common dependency. I spend about 10-20% of my time dealing with this sort of nonsense.
- bccdee 4y ago
  It sounds like your main issue is that you don't isolate your dependencies into virtual environments. That involves jumping through a couple hoops, sure (primarily just invoking `./venv/bin/activate` before you run the script), but for the most part it entirely resolves Python dependency problems. Could Python's dependency management be less clunky? Absolutely. But ultimately, when it comes to the tools you mentioned, the vendor is the one responsible for not explaining proper Python dependency management in their documentation, or for not creating an ergonomic wrapper around it.
- bb88 4y ago
  If you have issues like that, definitely look into virtualenvs and requirements.txt. It's meant to isolate python environments from breaking like that.

dangerface 4y ago

Tooling for embedded development? Please back in my day you just didn't get any tooling even the programer was left to the manufacture to make. > It is kind of like demanding you understand and set the timing advance on your car's engine. Why on earth would you even want to know of it, much less fiddle with it? Developers are not end users a technical understanding and capability is to be expected you aren't talking about the tooling for a pc you are talking about the tooling for a microcontroller. A more accurate example would be the timing belt manufacture expects the car manufacture to provide their own tooling to put the timing belt on their engine, in the same way a pic manufacture expects you to provide your own tooling. The end user is whoever buys the device you build with the pic in the same way that the end user of a timing belt is the person who buys the car not the manufacture that makes the car.