4 ms·

Python has dogshit package management that makes java look well thought out and sane.

echelon 2y ago
PYTHON @&$!! The Python community needs to solve this ASAP. This almost weekly pain point has turned a language I used to love in college into one of my most despised languages. The fact that the ML community uses this broken platform is infuriating. Make a Python version 4 that focuses only on fixing the packaging. 100% per-project hermeticy. No global packages whatsoever. Solve just this issue and bring the entire ecosystem on board. Kill all the various virtualenvs, the anacondas, global packages. All of it. Learn from Rust/Cargo. That project does it mostly right (sans lack of namespaces and reproducible builds).
- Eisenstein 2y ago
  It took 10 years to force a breaking migration from Python 2 to Python 3 in order to handle Unicode... I have resigned myself to repeatedly smashing my keyboard on the desk until it rains keys in blind rage induced frustration to let off steam and then calmly creating an issue in the appropriate repo instead of allowing even a modicum of hope that this will ever be fixed systemically.
- mr_toad 2y ago
  Wouldn’t this end up like the XKCD standards cartoon. You’d have yet another Python package system and even more fragmentation. https://xkcd.com/927/ https://xkcd.com/927/
  - ethbr1 2y ago
    If Python users weren't comfortable living in that reality already, they wouldn't still be Python users.
- superlopuh 2y ago
  As far as I'm concerned, this is the fix: https://github.com/astral-sh/uv https://github.com/astral-sh/uv
- jampekka 2y ago
  It's never going to be solved. The mess is deeply ingrained in the Python development culture. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/
1propionyl 2y ago
> that makes java look well thought out and sane That's not hard. Java's package management is well thought out and sane.
- lupire 2y ago
  Well, it's still PATH based, which is much worse than virtual environments (directory scoped) or Nix (well-versioned sharing).
pavlov 2y ago
Python's package management makes JavaScript look well thought out and sane. The bar is so low, you have to excavate it from a landfill of rotten node_modules...
- jampekka 2y ago
  Node.js has (or at least had before the ES6 mess) the sanest and best thought out package manager out there.
  - dumbo-octopus 2y ago
    NPM’s original sin was making package installation and general management so painless that folks installed micro packages for everything. Can’t really fault the software for that. The issue was, as you say, the introduction of ESM. It used to be that you required modules one way and one way only (yes there was AMD for advanced use cases, but it was an add-on), then people felt the need to “standardize” that, no we have this mess of ESM and CJS.
    - jampekka 2y ago
      I don't see what's wrong with micropackages.
      - VHRanger 2y ago
        Massive attack surface. That's true as a security risk, but also as a dependency risk (eg. left-pad).
        dumbo-octopus 2y ago
        If you don’t vendor your dependencies. Which is a poor practice that is commonly associated with NPM, but is by no means a requirement of the technology.
        2y ago
dvdkon 2y ago
I actually like Python's package management (with just pip and venv). It's different to all other modern solutions, taking a per-shell rather than per-project approach, but that doesn't mean worse. The advantage is that it's less "magic" than, say, npm. No need for rules saying how to find node_modules, you just have a PYTHONPATH env variable. The Rust and Java approach is to do everything through a build tool's CLI, and I can't complain about that. It's probably the best compromise: Less magic than npm, and more user-friendly than Python.
- jampekka 2y ago
  What magic is there for finding node_modules? It's in the same directory as package.json. Very similar (though simpler) to how Python modules work. There's also usually one global package dir, but that's almost exclusively used for runnable binaries, and not even really needed when there's npx. If you think Python packaging has less magic, you probably don't know it very deeply. I lost all hope for Python package management to ever get better when PEP 582 got rejected for some stupid tribal reasons. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/
  - dvdkon 2y ago
    I was talking more about the package lookup when running code, via e.g. `node script.js`. I think it looks in all parent directories of the CWD, or maybe of the script? It's not too complicated, but it is "more magic" IMO. Actually building Python packages is pretty complex, but that's the case for JS too. Java avoids this by distributing compiled libraries.
    - jampekka 2y ago
      It looks up from script.js'd directory or the earliest parent with node_modules. Not CWD. A lot like "import somemodule" in script.py tries to import somemodule.py file from the directory script.py is in. Traversing to the parent is especially nice for scripts. In python having scripts outside the module directory is quite painful. Gyp is a lot easier than Python setup.py. The "easy" Java packages are comparable to pure Python/pure JS packages. With e.g. C bindings JNI/Java packaging is a horrid pain.

PYTHON @&$!! The Python community needs to solve this ASAP. This almost weekly pain point has turned a language I used to love in college into one of my most despised languages. The fact that the ML community uses this broken platform is infuriating. Make a Python version 4 that focuses only on fixing the packaging. 100% per-project hermeticy. No global packages whatsoever. Solve just this issue and bring the entire ecosystem on board. Kill all the various virtualenvs, the anacondas, global packages. All of it. Learn from Rust/Cargo. That project does it mostly right (sans lack of namespaces and reproducible builds).

Eisenstein 2y ago
It took 10 years to force a breaking migration from Python 2 to Python 3 in order to handle Unicode... I have resigned myself to repeatedly smashing my keyboard on the desk until it rains keys in blind rage induced frustration to let off steam and then calmly creating an issue in the appropriate repo instead of allowing even a modicum of hope that this will ever be fixed systemically.
mr_toad 2y ago
Wouldn’t this end up like the XKCD standards cartoon. You’d have yet another Python package system and even more fragmentation. https://xkcd.com/927/ https://xkcd.com/927/
- ethbr1 2y ago
  If Python users weren't comfortable living in that reality already, they wouldn't still be Python users.
superlopuh 2y ago
As far as I'm concerned, this is the fix: https://github.com/astral-sh/uv https://github.com/astral-sh/uv
jampekka 2y ago
It's never going to be solved. The mess is deeply ingrained in the Python development culture. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/

1propionyl 2y ago

> that makes java look well thought out and sane That's not hard. Java's package management is well thought out and sane.

lupire 2y ago
Well, it's still PATH based, which is much worse than virtual environments (directory scoped) or Nix (well-versioned sharing).

pavlov 2y ago

Python's package management makes JavaScript look well thought out and sane. The bar is so low, you have to excavate it from a landfill of rotten node_modules...

jampekka 2y ago
Node.js has (or at least had before the ES6 mess) the sanest and best thought out package manager out there.
- dumbo-octopus 2y ago
  NPM’s original sin was making package installation and general management so painless that folks installed micro packages for everything. Can’t really fault the software for that. The issue was, as you say, the introduction of ESM. It used to be that you required modules one way and one way only (yes there was AMD for advanced use cases, but it was an add-on), then people felt the need to “standardize” that, no we have this mess of ESM and CJS.
  - jampekka 2y ago
    I don't see what's wrong with micropackages.
    - VHRanger 2y ago
      Massive attack surface. That's true as a security risk, but also as a dependency risk (eg. left-pad).
      - dumbo-octopus 2y ago
        If you don’t vendor your dependencies. Which is a poor practice that is commonly associated with NPM, but is by no means a requirement of the technology.
        2y ago

dvdkon 2y ago

I actually like Python's package management (with just pip and venv). It's different to all other modern solutions, taking a per-shell rather than per-project approach, but that doesn't mean worse. The advantage is that it's less "magic" than, say, npm. No need for rules saying how to find node_modules, you just have a PYTHONPATH env variable. The Rust and Java approach is to do everything through a build tool's CLI, and I can't complain about that. It's probably the best compromise: Less magic than npm, and more user-friendly than Python.

jampekka 2y ago
What magic is there for finding node_modules? It's in the same directory as package.json. Very similar (though simpler) to how Python modules work. There's also usually one global package dir, but that's almost exclusively used for runnable binaries, and not even really needed when there's npx. If you think Python packaging has less magic, you probably don't know it very deeply. I lost all hope for Python package management to ever get better when PEP 582 got rejected for some stupid tribal reasons. https://peps.python.org/pep-0582/ https://peps.python.org/pep-0582/
- dvdkon 2y ago
  I was talking more about the package lookup when running code, via e.g. `node script.js`. I think it looks in all parent directories of the CWD, or maybe of the script? It's not too complicated, but it is "more magic" IMO. Actually building Python packages is pretty complex, but that's the case for JS too. Java avoids this by distributing compiled libraries.
  - jampekka 2y ago
    It looks up from script.js'd directory or the earliest parent with node_modules. Not CWD. A lot like "import somemodule" in script.py tries to import somemodule.py file from the directory script.py is in. Traversing to the parent is especially nice for scripts. In python having scripts outside the module directory is quite painful. Gyp is a lot easier than Python setup.py. The "easy" Java packages are comparable to pure Python/pure JS packages. With e.g. C bindings JNI/Java packaging is a horrid pain.