5 ms·

Why does everything related to versioning and packaging have to be so difficult with Python? I never understood that.

thenewwazoo 6y ago
I had a whole comment written up saying basically this, but in less-kind terms. Python just seems so incredibly brittle.
artful-hacker 6y ago
I'd hazard a guess that it is because Python is so old. It was literally created in the 80s. Package management, explosion of dependencies, multiple implementations of the interpreter, being added to the base operating system, and all that fancy stuff came after. I remember doing versioning and packaging in C++ in early 2000s, and it being just as annoying but for different reasons. People are trying to fix it, and their are some great packages out there that make it easier. Pyenv, Poetry, this asdf tool the fine article mentions, all seem like good solutions.
- coldtea 6y ago
  >I'd hazard a guess that it is because Python is so old. It was literally created in the 80s. Package management, explosion of dependencies, multiple implementations of the interpreter, being added to the base operating system, and all that fancy stuff came after. So? You can still create nice tools for it, even decades after a language has been created...
acdha 6y ago
It's not. People arbitrarily complicate their personal workflows, especially having years of habit predating the existence of modern tools, but this is a voluntary decision to make local development complicated and you can choose not to work that way. As an example, any time Homebrew ships a new update I run “pipx reinstall-all” and, for local projects, I let pipenv / poetry rebuild the virtualenv the next time I use it. This is automated and takes far less of my time than, say, managing dependency updates.
- ilikepi 6y ago
  I think part of the challenge (from my perspective, as someone who only rarely needs to mess with python) is not that the workflow you describe is particularly complicated in isolation, it's the shear number of different tools that ostensibly have the same goal: to manage python virtual environments and packages. Because there are so many different tools, there is a near-limitless supply of different articles and SO posts in which people describe the "best" way to accomplish the most basic setup and maintenance of a virtual environment. It's quite difficult for new and/or occasional users to choose a good way to get started. Even if there is an "official" way documented on python.org for each Python version, when you run into a problem with it, inevitably you're stuck filtering through lots of advice to "just use <third-party tool xyz> instead".
- draw_down 6y ago
  Ah, my mistake! I had thought it was a huge problem and a constant pain in the ass, but it turns out that actually everyone is just stupid. Whew, what a relief.
- coldtea 6y ago
  >As an example, any time Homebrew ships a new update I run “pipx reinstall-all” and, for local projects, I let pipenv / poetry rebuild the virtualenv the next time I use it. This is automated and takes far less of my time than, say, managing dependency updates. So, giving up (and re-installing everything from scratch or rebuilding virtualenvs, using several third party tools (pipx on top of pip, pipenv and/or poetry in lieu/in place of virtualenv) is your argument to how "it's not complicated" and people merely "arbitrarily complicate their personal workflows"? Oh, the irony!
  - acdha 6y ago
    The key part which you missed is that everything is automated, always installed in an isolated virtual environment, the package are automatically tracked and updated, and the tools which make this happen are installed from Homebrew and update at the same time. As to why there are two tools, it's because they have separate goals: pipx provides a robust way to manage global installs of Python utilities in per-tool virtualenvs while tools like poetry or pipenv are aimed at projects you're developing and handle things like automating the process of installing updates & updating the corresponding metadata files so you can ship clean updates. All of them use virtualenvs and pip underneath — it's just a modern workflow. The key part is recognizing that the problem comes from trying to fight updates: software changes and accepting that means moving to a workflow which automates that rather than spending time trying to freeze everything in amber. Learning to use a tool every decade or so is a LOT less time consuming than doing this by hand. The average poster in this thread has already spent more time on this topic than my projects have needed for the mechanics of installing Python packages in any given year.
qbasic_forever 6y ago
Python was created in the 90s when most unix users were on huge minicomputer-like shared systems (mostly at universities). In those days you had entire IT staff dedicated to curating and maintaining all of the software on the system, making sure it ran well, keeping it updated, etc. Guido explicitly did not want to bog the language development down with time spent on building packaging, deployment, etc.--those were matters for the system admins. We've all kind of been dealing with the consequences of that decision for many years.

I had a whole comment written up saying basically this, but in less-kind terms. Python just seems so incredibly brittle.

artful-hacker 6y ago

I'd hazard a guess that it is because Python is so old. It was literally created in the 80s. Package management, explosion of dependencies, multiple implementations of the interpreter, being added to the base operating system, and all that fancy stuff came after. I remember doing versioning and packaging in C++ in early 2000s, and it being just as annoying but for different reasons. People are trying to fix it, and their are some great packages out there that make it easier. Pyenv, Poetry, this asdf tool the fine article mentions, all seem like good solutions.

coldtea 6y ago
>I'd hazard a guess that it is because Python is so old. It was literally created in the 80s. Package management, explosion of dependencies, multiple implementations of the interpreter, being added to the base operating system, and all that fancy stuff came after. So? You can still create nice tools for it, even decades after a language has been created...

acdha 6y ago

It's not. People arbitrarily complicate their personal workflows, especially having years of habit predating the existence of modern tools, but this is a voluntary decision to make local development complicated and you can choose not to work that way. As an example, any time Homebrew ships a new update I run “pipx reinstall-all” and, for local projects, I let pipenv / poetry rebuild the virtualenv the next time I use it. This is automated and takes far less of my time than, say, managing dependency updates.

ilikepi 6y ago
I think part of the challenge (from my perspective, as someone who only rarely needs to mess with python) is not that the workflow you describe is particularly complicated in isolation, it's the shear number of different tools that ostensibly have the same goal: to manage python virtual environments and packages. Because there are so many different tools, there is a near-limitless supply of different articles and SO posts in which people describe the "best" way to accomplish the most basic setup and maintenance of a virtual environment. It's quite difficult for new and/or occasional users to choose a good way to get started. Even if there is an "official" way documented on python.org for each Python version, when you run into a problem with it, inevitably you're stuck filtering through lots of advice to "just use <third-party tool xyz> instead".
draw_down 6y ago
Ah, my mistake! I had thought it was a huge problem and a constant pain in the ass, but it turns out that actually everyone is just stupid. Whew, what a relief.
coldtea 6y ago
>As an example, any time Homebrew ships a new update I run “pipx reinstall-all” and, for local projects, I let pipenv / poetry rebuild the virtualenv the next time I use it. This is automated and takes far less of my time than, say, managing dependency updates. So, giving up (and re-installing everything from scratch or rebuilding virtualenvs, using several third party tools (pipx on top of pip, pipenv and/or poetry in lieu/in place of virtualenv) is your argument to how "it's not complicated" and people merely "arbitrarily complicate their personal workflows"? Oh, the irony!
- acdha 6y ago
  The key part which you missed is that everything is automated, always installed in an isolated virtual environment, the package are automatically tracked and updated, and the tools which make this happen are installed from Homebrew and update at the same time. As to why there are two tools, it's because they have separate goals: pipx provides a robust way to manage global installs of Python utilities in per-tool virtualenvs while tools like poetry or pipenv are aimed at projects you're developing and handle things like automating the process of installing updates & updating the corresponding metadata files so you can ship clean updates. All of them use virtualenvs and pip underneath — it's just a modern workflow. The key part is recognizing that the problem comes from trying to fight updates: software changes and accepting that means moving to a workflow which automates that rather than spending time trying to freeze everything in amber. Learning to use a tool every decade or so is a LOT less time consuming than doing this by hand. The average poster in this thread has already spent more time on this topic than my projects have needed for the mechanics of installing Python packages in any given year.

qbasic_forever 6y ago

Python was created in the 90s when most unix users were on huge minicomputer-like shared systems (mostly at universities). In those days you had entire IT staff dedicated to curating and maintaining all of the software on the system, making sure it ran well, keeping it updated, etc. Guido explicitly did not want to bog the language development down with time spent on building packaging, deployment, etc.--those were matters for the system admins. We've all kind of been dealing with the consequences of that decision for many years.