5 ms·

>I don't think any of this precludes immutability: my understanding is Debian could package every version variant (or find common variants without violating sem

>I don't think any of this precludes immutability: my understanding is Debian could package every version variant (or find common variants without violating semver) and maintain both immutability and their global view. Debian is a binary-first distro so this would obligate them to produce probably 5x the binary packages for the same thing. Then you have higher chances of conflicts, unless I'm missing something. C and C++ shared libraries support coexistence of multiple versions via semver-based name schemes. I don't know if Rust packages are structured that well. >Empirically, not that badly. It's what tools like `uv` and `pipx` do by default, and it results in a markedly better net user experience (since Python tools actually behave like hermetic tools, and not implicit modifiers of global resolution state). It's also what Homebrew does -- every packaged Python formula in Homebrew gets shipped in its own virtual environment. These are typically not used to install everything that goes into a whole desktop or server operating system. They're used to install a handful of applications that the user wants. If you want to support as many systems as possible, you need to be mindful of resource usage. >I would argue that ignoring upstream semver constraints challenges the overall project :-) Yes it's a horrible idea. "Let's programmatically add a ton of bugs and wait for victims to report the bugs back to us in the future" is what I'm reading. A policy like that can be exploited by malicious actors. At minimum they need to ship the correct required versions of everything, if they ship anything.

woodruffw 2y ago
> Debian is a binary-first distro so this would obligate them to produce probably 5x the binary packages for the same thing. Then you have higher chances of conflicts, unless I'm missing something. Ah yeah, this wouldn't work -- instead, Debian would need to bite the bullet on Rust preferring static linkage and accept that each package might have different interior dependencies (still static and known, just not globally consistent). This doesn't represent a conflict risk because of the static linkage, but it's very much against Debian's philosophy (as I understand it). > I don't know if Rust packages are structured that well. Rust packages of different versions can gracefully coexist (they do already at the crate resolution level), but static linkage is the norm. > These are typically not used to install everything that goes into a whole desktop or server operating system. They're used to install a handful of applications that the user wants. I might not be understanding what you mean, but I don't think the user/machine distinction is super relevant in most deployments: in practice the server's software shouldn't be running as root anyways, so it doesn't matter much that it's installed in a user-held virtual environment. And with respect to resource consumption: unless I'm missing something, I think the resource difference between installing a stack with `pip` and installing that same stack with `apt` should be pretty marginal -- installers will pay a linear cost for each new virtual environment, but I can't imagine that being a dealbreaker in most setups (already multiple venvs are atypical, and you'd have to be pretty constrained in terms of storage space to have issues with a few duplicate installs of `requests` or similar).
- wakawaka28 2y ago
  >I might not be understanding what you mean, but I don't think the user/machine distinction is super relevant in most deployments: in practice the server's software shouldn't be running as root anyways, so it doesn't matter much that it's installed in a user-held virtual environment. Many software packages need root access but that is not what I was talking about. Distro users just want working software with minimal resource usage and incompatibilities. >Rust packages of different versions can gracefully coexist (they do already at the crate resolution level), but static linkage is the norm. Static linkage is deliberately avoided as much as possible by distros like Debian due to the additional overhead. It's overhead on the installation side and mega overhead on the server that has to host a download of essentially the same dependency many times for each installation when it could have instead been downloaded once. >And with respect to resource consumption: unless I'm missing something, I think the resource difference between installing a stack with `pip` and installing that same stack with `apt` should be pretty marginal -- installers will pay a linear cost for each new virtual environment, but I can't imagine that being a dealbreaker in most setups (already multiple venvs are atypical, and you'd have to be pretty constrained in terms of storage space to have issues with a few duplicate installs of `requests` or similar). If the binary package is a thin wrapper around venv, then you're right. But these packages are usually designed to share dependencies with other packages where possible. So for example, if you had two packages installed using some huge library for example, they only need one copy of that library between them. Updating the library only requires downloading a new version of the library. Updating the library if it is statically linked requires downloading it twice along with the other code it's linked with, potentially using many times the amount of resources on network and disk. Static linking is convenient sometimes but it isn't free.
  - int_19h 2y ago
    Historically, the main reason why dynamic linking is even a thing is because RAM was too limited to run "heavy" software like, say, an X server. This hasn't been true for decades now.
    - wakawaka28 2y ago
fragmede 2y ago
> Then you have higher chances of conflicts, unless I'm missing something. For python, you could install libraries into a versioned dir, and then create a venv for each program, and then in each venv/lib/pythonX/site-packages/libraryY dir just symlinks to the appropriate versioned global copy.
- wakawaka28 2y ago
  That would make it difficult to tell at a system level what the exact installed dependencies of a program are. It would also require the distro to basically re-invent pip. Want to invoke one venv program from another one? Well, good luck figuring out conflicts in their environments which can be incompatible from the time they are installed. Now you're talking about a wrapper for each program just to load the right settings. This is not even an exhaustive list of all possible complications that are solved by having one global set of packages.
- LtWorf 2y ago
  Do you think that's user friendly?
  - fragmede 2y ago
    I see it as more user friendly - instead of forgetting to activate the venv and having the program fail to run/be broken/act weird, you run the program and it activates the venv for you so you don't have that problem.
    - LtWorf 2y ago
      Do you think your software is so important that people will do all of that rather than use something better? (For example your software but patched by a distribution to work easily without doing all of that complication)
      - fragmede 2y ago
        I'm talking about a shim that distributions can use to launch python programs, some which they distribute, rather than software I write. In particular, ML researchers aren't sysadmins but a lot of their software is in the form of community python programs, which is to say not polished commerical apps with backing like, say, Adobe Photoshop, and this shim solves one of python's pitfalls for users.

woodruffw 2y ago

> Debian is a binary-first distro so this would obligate them to produce probably 5x the binary packages for the same thing. Then you have higher chances of conflicts, unless I'm missing something. Ah yeah, this wouldn't work -- instead, Debian would need to bite the bullet on Rust preferring static linkage and accept that each package might have different interior dependencies (still static and known, just not globally consistent). This doesn't represent a conflict risk because of the static linkage, but it's very much against Debian's philosophy (as I understand it). > I don't know if Rust packages are structured that well. Rust packages of different versions can gracefully coexist (they do already at the crate resolution level), but static linkage is the norm. > These are typically not used to install everything that goes into a whole desktop or server operating system. They're used to install a handful of applications that the user wants. I might not be understanding what you mean, but I don't think the user/machine distinction is super relevant in most deployments: in practice the server's software shouldn't be running as root anyways, so it doesn't matter much that it's installed in a user-held virtual environment. And with respect to resource consumption: unless I'm missing something, I think the resource difference between installing a stack with `pip` and installing that same stack with `apt` should be pretty marginal -- installers will pay a linear cost for each new virtual environment, but I can't imagine that being a dealbreaker in most setups (already multiple venvs are atypical, and you'd have to be pretty constrained in terms of storage space to have issues with a few duplicate installs of `requests` or similar).

wakawaka28 2y ago
>I might not be understanding what you mean, but I don't think the user/machine distinction is super relevant in most deployments: in practice the server's software shouldn't be running as root anyways, so it doesn't matter much that it's installed in a user-held virtual environment. Many software packages need root access but that is not what I was talking about. Distro users just want working software with minimal resource usage and incompatibilities. >Rust packages of different versions can gracefully coexist (they do already at the crate resolution level), but static linkage is the norm. Static linkage is deliberately avoided as much as possible by distros like Debian due to the additional overhead. It's overhead on the installation side and mega overhead on the server that has to host a download of essentially the same dependency many times for each installation when it could have instead been downloaded once. >And with respect to resource consumption: unless I'm missing something, I think the resource difference between installing a stack with `pip` and installing that same stack with `apt` should be pretty marginal -- installers will pay a linear cost for each new virtual environment, but I can't imagine that being a dealbreaker in most setups (already multiple venvs are atypical, and you'd have to be pretty constrained in terms of storage space to have issues with a few duplicate installs of `requests` or similar). If the binary package is a thin wrapper around venv, then you're right. But these packages are usually designed to share dependencies with other packages where possible. So for example, if you had two packages installed using some huge library for example, they only need one copy of that library between them. Updating the library only requires downloading a new version of the library. Updating the library if it is statically linked requires downloading it twice along with the other code it's linked with, potentially using many times the amount of resources on network and disk. Static linking is convenient sometimes but it isn't free.
- int_19h 2y ago
  Historically, the main reason why dynamic linking is even a thing is because RAM was too limited to run "heavy" software like, say, an X server. This hasn't been true for decades now.
  - wakawaka28 2y ago

fragmede 2y ago

> Then you have higher chances of conflicts, unless I'm missing something. For python, you could install libraries into a versioned dir, and then create a venv for each program, and then in each venv/lib/pythonX/site-packages/libraryY dir just symlinks to the appropriate versioned global copy.

wakawaka28 2y ago
That would make it difficult to tell at a system level what the exact installed dependencies of a program are. It would also require the distro to basically re-invent pip. Want to invoke one venv program from another one? Well, good luck figuring out conflicts in their environments which can be incompatible from the time they are installed. Now you're talking about a wrapper for each program just to load the right settings. This is not even an exhaustive list of all possible complications that are solved by having one global set of packages.
LtWorf 2y ago
Do you think that's user friendly?
- fragmede 2y ago
  I see it as more user friendly - instead of forgetting to activate the venv and having the program fail to run/be broken/act weird, you run the program and it activates the venv for you so you don't have that problem.
  - LtWorf 2y ago
    Do you think your software is so important that people will do all of that rather than use something better? (For example your software but patched by a distribution to work easily without doing all of that complication)
    - fragmede 2y ago
      I'm talking about a shim that distributions can use to launch python programs, some which they distribute, rather than software I write. In particular, ML researchers aren't sysadmins but a lot of their software is in the form of community python programs, which is to say not polished commerical apps with backing like, say, Adobe Photoshop, and this shim solves one of python's pitfalls for users.