6 ms·

And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applicatio

And yet, other languages ship package managers [1] that are widely loved within their ecosystems and cover everything from microcontrollers to server applications. I think what it makes it more difficult in the case of Python is that it has decades of legacy to deal with. No consistent semantic versioning, packages that expect that they can modify their package path in-place (this is a nightmare for immutable systems, like NixOS or OSTree-based system like Silverblue), a wide variety of build systems that sometimes hook into make, etc. Solving this is a hard problem. This is why an authority like PSF has to step in and say: this is how it is going to be done from here onwards. [1] E.g. Rust's Cargo.

komuher 5y ago
Kinda diffrent Rust its like 3 years old when it start getting popular. Python is 13+ years old when it starts getting very popular. You cannot compare legacy to modern solution -.-
- smallerfish 5y ago
  Maven was launched 9 years after Java got popular. Everybody was using Ant, everybody decided Ant sucked and moved to the new solution. While Gradle is the new kid on the block, it keeps the infrastructure of dependency management and is thus compatible with Maven to that extent. It can be done.
  - RegW 5y ago
    Ant didn't do package management.
    - smallerfish 5y ago
      No, but ivy was commonly used. That said, the bigger point is that the standard way to do things can be changed. For some reason, despite (or because of?) PEP, the python community seems unable to coalesce around standard ways of packaging.
      - RegW 5y ago
        Perhaps I have lived too long out in the provences, but Maven was my first experience of dependency management in Java. After Ant it was a no-brainer because Ant didn't do dependencies. I didn't come across references to Ivy for some years and personally I have never seen it used in the wild. That said, I didn't actually like early Maven that much, it was pretty inflexible and often required the AntRun plugin to do something novel. However, it is still top dog everywhere I work and very much the standard way to do things.
- WhyNotHugo 5y ago
  Which is why I'm starting to feel like Python is the new Java.
  - oneweekwonder 5y ago
    Strange, because python was released almost 5 years before java.
    - mangecoeur 5y ago
      Python is the Old Java :P
- microtonal 5y ago
  Which is exactly what my second paragraph said :). This is a common struggle for many older programming language ecosystems, e.g. I think the same is true to some extend for C and C++. As one of the sibling commenters mentions, there are good examples that it is possible to standardize packaging better. Maven replaced IDE-driven builds and Ant for in the Java ecosystem and added proper package management. Additionally, it required that projects start conforming to standardized layouts, by taking convention over configuration and being largely declarative. I think the Maven success story lost some of its shine with Gradle, but that's another story.
- ajsnigrutin 5y ago
  Perl is 33 years old, and perl 5.x is 27 years old. I can literally take a 20+yo book and all the examples still work. CPAN still works. I literally have 20+yo scripts still copied from server to server, from laptop to laptop, without any changes.
__float 5y ago
Is it fair to compare an interpreted language and its package manager to Rust and Cargo? Python packages ship their source (in most cases), depend on a locally installed interpreter (with semantics possibly changing by version). Yes, Python packages often make poor assumptions about what setup.py can do (i.e., _anything_), and so you end up choosing between "tested, supported by the author, and old" or "untested, unsupported, but up to date".
- wildbook 5y ago
  Rust almost does the exact same thing, so I'd say it's fair. Dependencies (crates) are grabbed in source format and compiled locally as part of the build process, and installing Rust programs through Cargo also compiles them (and their dependencies) locally. Some crates have the same issue where build scripts rely on outside tooling being installed, but it's definitely not common to (unless you're relying on compiling C/C++ code for FFI for example, in which case it's somewhat frequent).
  - __float 5y ago
    I think there needs to be a distinction between fetching crates as source at _build time_ and what happens with Python. Python's "build time" will still require the source to be present on the target machine it's deployed to -- unfortunately these tools are complex because "packaging" is only half of the issue, it's also the distribution and _deployment_ which makes things messy. Consider that, even if you want to use packages only from your application's virtualenv, the default (footgun warning!) is that Python will still use the "system" packages -- this means you may have installed Ansible or some other tool that relies on Python and many packages from the distro package manager. But your app could pick up one of those dependencies! At best, this will work fine. But in the worse cases, perhaps it subtly behaves differently or simply does not work at all. My understanding is that Rust will, by default, statically link all of these dependencies. This, in Python, would be like a "pex" or "par" (or one of the many other options :^)), which does make the distribution aspect much simpler. (At the cost of build-time complexity, slowdowns, and occasional incompatibility.)
- regularfry 5y ago
turminal 5y ago
Cargo is a good example of a survivorship bias of sorts. It's so deeply integrated into rust that if you can't stomach it you just leave the ecosystem and everyone that remains likes it ;)
- pdimitar 5y ago
  Hm, so what did you dislike about `cargo`?
uranusjr 5y ago
Rust has and “advantage” here that it’s not generally shipped with your distro’s package manager. I think my biggest problem with this article is that the distros put Python there in the first place, and all of them apply patches to make Python work how they want it. And when it doesn’t work, it’s Python’s fault…? I mean yeah Python can definitely do something to make distributing Python easier, but it can only do so much without distros’ direct involvement. I’ll add that most of Linux distro packaging contributors are generally very nice people, understand the problem at hand, and are very open to collaboration. But sometimes you see this kind of “it’s all your fault” complaints and it’s doing exactly the opposite of helping the cause.
- drran 5y ago
  Available Packages Name : rust Version : 1.56.1 Release : 1.fc34
mangecoeur 5y ago
Worth noting that the PSF has no authority to tell people how to do their packaging. Rust's Cargo is 20 years newer than Python and benefits from those decades of experience. It's a very different proposition to start a new system from fresh than to try to migrate a huge and diverse community towards it.
- regularfry 5y ago
  This is actually where a BDFL should step in. "We're doing X for Python 5, we're using it to manage the stdlib as well as community packages, everyone get ready because I have decided."
  - Macha 5y ago
    They tried this with pip. Distros ripped it out. They added the ensurepip module so users could add it after the fact. Distros ripped it out too. Arguably the advantage newer languages like Rust and Go have in this regard is they don't even consider the distro use case - you're going to get static linking and you'd better like it. Whereas Python is from an older era and tries to fit in with the local customs and so gets hammered for its inconsistency
    - regularfry 5y ago
      Yes, and that's fine. Distro python isn't for end users.
thetwentyone 5y ago
And Julia's Pkg! One of my primary reasons for switching away from Python to Julia was how ergonomic modern dependency management was.

komuher 5y ago

Kinda diffrent Rust its like 3 years old when it start getting popular. Python is 13+ years old when it starts getting very popular. You cannot compare legacy to modern solution -.-

smallerfish 5y ago
Maven was launched 9 years after Java got popular. Everybody was using Ant, everybody decided Ant sucked and moved to the new solution. While Gradle is the new kid on the block, it keeps the infrastructure of dependency management and is thus compatible with Maven to that extent. It can be done.
- RegW 5y ago
  Ant didn't do package management.
  - smallerfish 5y ago
    No, but ivy was commonly used. That said, the bigger point is that the standard way to do things can be changed. For some reason, despite (or because of?) PEP, the python community seems unable to coalesce around standard ways of packaging.
    - RegW 5y ago
      Perhaps I have lived too long out in the provences, but Maven was my first experience of dependency management in Java. After Ant it was a no-brainer because Ant didn't do dependencies. I didn't come across references to Ivy for some years and personally I have never seen it used in the wild. That said, I didn't actually like early Maven that much, it was pretty inflexible and often required the AntRun plugin to do something novel. However, it is still top dog everywhere I work and very much the standard way to do things.
WhyNotHugo 5y ago
Which is why I'm starting to feel like Python is the new Java.
- oneweekwonder 5y ago
  Strange, because python was released almost 5 years before java.
  - mangecoeur 5y ago
    Python is the Old Java :P
microtonal 5y ago
Which is exactly what my second paragraph said :). This is a common struggle for many older programming language ecosystems, e.g. I think the same is true to some extend for C and C++. As one of the sibling commenters mentions, there are good examples that it is possible to standardize packaging better. Maven replaced IDE-driven builds and Ant for in the Java ecosystem and added proper package management. Additionally, it required that projects start conforming to standardized layouts, by taking convention over configuration and being largely declarative. I think the Maven success story lost some of its shine with Gradle, but that's another story.
ajsnigrutin 5y ago
Perl is 33 years old, and perl 5.x is 27 years old. I can literally take a 20+yo book and all the examples still work. CPAN still works. I literally have 20+yo scripts still copied from server to server, from laptop to laptop, without any changes.

__float 5y ago

Is it fair to compare an interpreted language and its package manager to Rust and Cargo? Python packages ship their source (in most cases), depend on a locally installed interpreter (with semantics possibly changing by version). Yes, Python packages often make poor assumptions about what setup.py can do (i.e., _anything_), and so you end up choosing between "tested, supported by the author, and old" or "untested, unsupported, but up to date".

wildbook 5y ago
Rust almost does the exact same thing, so I'd say it's fair. Dependencies (crates) are grabbed in source format and compiled locally as part of the build process, and installing Rust programs through Cargo also compiles them (and their dependencies) locally. Some crates have the same issue where build scripts rely on outside tooling being installed, but it's definitely not common to (unless you're relying on compiling C/C++ code for FFI for example, in which case it's somewhat frequent).
- __float 5y ago
  I think there needs to be a distinction between fetching crates as source at _build time_ and what happens with Python. Python's "build time" will still require the source to be present on the target machine it's deployed to -- unfortunately these tools are complex because "packaging" is only half of the issue, it's also the distribution and _deployment_ which makes things messy. Consider that, even if you want to use packages only from your application's virtualenv, the default (footgun warning!) is that Python will still use the "system" packages -- this means you may have installed Ansible or some other tool that relies on Python and many packages from the distro package manager. But your app could pick up one of those dependencies! At best, this will work fine. But in the worse cases, perhaps it subtly behaves differently or simply does not work at all. My understanding is that Rust will, by default, statically link all of these dependencies. This, in Python, would be like a "pex" or "par" (or one of the many other options :^)), which does make the distribution aspect much simpler. (At the cost of build-time complexity, slowdowns, and occasional incompatibility.)
regularfry 5y ago

turminal 5y ago

Cargo is a good example of a survivorship bias of sorts. It's so deeply integrated into rust that if you can't stomach it you just leave the ecosystem and everyone that remains likes it ;)

pdimitar 5y ago
Hm, so what did you dislike about `cargo`?

uranusjr 5y ago

Rust has and “advantage” here that it’s not generally shipped with your distro’s package manager. I think my biggest problem with this article is that the distros put Python there in the first place, and all of them apply patches to make Python work how they want it. And when it doesn’t work, it’s Python’s fault…? I mean yeah Python can definitely do something to make distributing Python easier, but it can only do so much without distros’ direct involvement. I’ll add that most of Linux distro packaging contributors are generally very nice people, understand the problem at hand, and are very open to collaboration. But sometimes you see this kind of “it’s all your fault” complaints and it’s doing exactly the opposite of helping the cause.

drran 5y ago
Available Packages Name : rust Version : 1.56.1 Release : 1.fc34

mangecoeur 5y ago

Worth noting that the PSF has no authority to tell people how to do their packaging. Rust's Cargo is 20 years newer than Python and benefits from those decades of experience. It's a very different proposition to start a new system from fresh than to try to migrate a huge and diverse community towards it.

regularfry 5y ago
This is actually where a BDFL should step in. "We're doing X for Python 5, we're using it to manage the stdlib as well as community packages, everyone get ready because I have decided."
- Macha 5y ago
  They tried this with pip. Distros ripped it out. They added the ensurepip module so users could add it after the fact. Distros ripped it out too. Arguably the advantage newer languages like Rust and Go have in this regard is they don't even consider the distro use case - you're going to get static linking and you'd better like it. Whereas Python is from an older era and tries to fit in with the local customs and so gets hammered for its inconsistency
  - regularfry 5y ago
    Yes, and that's fine. Distro python isn't for end users.

thetwentyone 5y ago

And Julia's Pkg! One of my primary reasons for switching away from Python to Julia was how ergonomic modern dependency management was.