5 ms·

Python's package ecosystem is probably the worst in the world

artemonster 3y ago
Can you elaborate what this system lacks or what are the key properties that make it „the worst“? I know its a mess, but I cant pinpoint the reason why.
- kristofferR 3y ago
  That so many things doesn't run on the latest stable version is one of them. Python 3.11 was published in all the way back in October, and it's still not stable and well supported.
- linkdd 3y ago
  Is the fact that there are multiple package managers for the same package fornat/spec the reason why people say it's a mess? Genuine question: How is PyPI different from NPM, Maven, crates.io, etc..? Why "Python packaging is a mess" and not the others?
- throwawaaarrgh 3y ago
  The non-hierarchical, not-reused, reinvented package convention. The most popular package repository being incredibly hard to find one package you want among a sea of crap. The lack of a single standard solution to the most common problems. Frequent backwards compatibility issues. It's definitely not the worst. But it's hilarious how bad it is considering how popular it is and how long it's been around
j16sdiz 3y ago
try ruby.
- Alifatisk 3y ago
  What's wrong with bundler / gem?
  - vocworker 3y ago
    System Ruby basically breaks "gem install", right? However once you install your own Ruby as a user, it's easy to use. Interestingly, Ruby was the first language to improve on Perl's CPAN. Maven 2 was inspired by it. For their age, bundler/gem do quite well.
- jeltz 3y ago
  Yes, it is way better which is why many layer package managers, e.g. cargo and composer, took the ideas from Ruby to improve on. Ruby has issues but the package management is not one of them.
rcfox 3y ago
Worse than Javascript's 5+ different types of mostly incompatible module formats, transpilers for different version targets, megalithic node_modules directories and the community's love of importing new packages for one-line functions that can disappear and take down the internet?
- linkdd 3y ago
  How is `node_modules` different from `/path/to/venv/lib/python3.X/site-packages` ? Are PyPI, cargo, Maven, etc... all immune to supply chain attacks?
  - rcfox 3y ago
    One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself. Python also has "wheels", which are pre-compiled distributions, rather than full-source distributions. These are often smaller and install faster.
    - heywhatupboys 3y ago
      > One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself. a feature not a bug. Anyone who ahs built a sufficiently large project and had to abandon some dependencies know this pain
- jeltz 3y ago
  Yes. Which is quite impressive.
- deleted 3y ago
  [deleted]
- whywhywhywhy 3y ago
  As much as we all have criticized the npm ecosystem, after seeing what exists in the python world I actually came round to it and consider it very good.
wongarsu 3y ago
In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library. Until you reach sufficient scale, at which point the sane thing seems to be to make your own build system instead
- logicchains 3y ago
  >In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library. That's from the developer's perspective, not the user's. The user of a C++ project doesn't have to care about any of that; they just run the binary, or ./configure && make to build from source. The user of a Python project has to struggle through worst case days of pain depending on how many dependencies are randomly broken.
  - wongarsu 3y ago
    If your user is interacting with the package ecosystem, you already failed multiple steps. Which python projects are routinely doing, but that's a cultural issue about packaging. For "users" that have to touch the code: well, those get the full developer's perspective. And I've lost more hours than I would like to admit trying to get C++ projects to build. A project has to be really small or packaged really well to make it as simple as ./configure && make

Can you elaborate what this system lacks or what are the key properties that make it „the worst“? I know its a mess, but I cant pinpoint the reason why.

kristofferR 3y ago
That so many things doesn't run on the latest stable version is one of them. Python 3.11 was published in all the way back in October, and it's still not stable and well supported.
linkdd 3y ago
Is the fact that there are multiple package managers for the same package fornat/spec the reason why people say it's a mess? Genuine question: How is PyPI different from NPM, Maven, crates.io, etc..? Why "Python packaging is a mess" and not the others?
throwawaaarrgh 3y ago
The non-hierarchical, not-reused, reinvented package convention. The most popular package repository being incredibly hard to find one package you want among a sea of crap. The lack of a single standard solution to the most common problems. Frequent backwards compatibility issues. It's definitely not the worst. But it's hilarious how bad it is considering how popular it is and how long it's been around

j16sdiz 3y ago

try ruby.

Alifatisk 3y ago
What's wrong with bundler / gem?
- vocworker 3y ago
  System Ruby basically breaks "gem install", right? However once you install your own Ruby as a user, it's easy to use. Interestingly, Ruby was the first language to improve on Perl's CPAN. Maven 2 was inspired by it. For their age, bundler/gem do quite well.
jeltz 3y ago
Yes, it is way better which is why many layer package managers, e.g. cargo and composer, took the ideas from Ruby to improve on. Ruby has issues but the package management is not one of them.

rcfox 3y ago

Worse than Javascript's 5+ different types of mostly incompatible module formats, transpilers for different version targets, megalithic node_modules directories and the community's love of importing new packages for one-line functions that can disappear and take down the internet?

linkdd 3y ago
How is `node_modules` different from `/path/to/venv/lib/python3.X/site-packages` ? Are PyPI, cargo, Maven, etc... all immune to supply chain attacks?
- rcfox 3y ago
  One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself. Python also has "wheels", which are pre-compiled distributions, rather than full-source distributions. These are often smaller and install faster.
  - heywhatupboys 3y ago
    > One major difference is a venv doesn't allow nested dependencies. With npm, you might install version 2.3 of a package, only to find that some other package has installed version 1.4 under itself. a feature not a bug. Anyone who ahs built a sufficiently large project and had to abandon some dependencies know this pain
jeltz 3y ago
Yes. Which is quite impressive.
deleted 3y ago
[deleted]
whywhywhywhy 3y ago
As much as we all have criticized the npm ecosystem, after seeing what exists in the python world I actually came round to it and consider it very good.

wongarsu 3y ago

In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library. Until you reach sufficient scale, at which point the sane thing seems to be to make your own build system instead

logicchains 3y ago
>In Python at least you "only" have to figure out venv, pip and conda and you're mostly set. In C/C++ world adding a dependency is such a pain that most projects have less than five dependencies. Most of the time it's simpler to just quickly write and debug a worse version yourself than to figure out how to find and integrate a library. That's from the developer's perspective, not the user's. The user of a C++ project doesn't have to care about any of that; they just run the binary, or ./configure && make to build from source. The user of a Python project has to struggle through worst case days of pain depending on how many dependencies are randomly broken.
- wongarsu 3y ago
  If your user is interacting with the package ecosystem, you already failed multiple steps. Which python projects are routinely doing, but that's a cultural issue about packaging. For "users" that have to touch the code: well, those get the full developer's perspective. And I've lost more hours than I would like to admit trying to get C++ projects to build. A project has to be really small or packaged really well to make it as simple as ./configure && make