7 ms·

> Get rid of the requirement that there is only one stable (minor) version of a package in the distribution at one time. I think this requirement made sense wh

> Get rid of the requirement that there is only one stable (minor) version of a package in the distribution at one time. I think this requirement made sense when disk space was scarce. I think this requirement makes sense if you trust that your distro is always better at choosing the 'best' version of a dependency that some software should use than the software author. Nowadays, I think neither is generally true. Disk is plentiful, distro packages are almost always far more out of date than the software's original source, and allowing authors to ship software with specific pinned dependency versions reduces bugs caused by dependency changes and makes providing support for software (and especially reproducing end-user's issues) significantly easier. Isolating dependencies by application, with linking to avoid outright duplication of identical versions (a la pnpm's approach for JS: https://pnpm.io/ https://pnpm.io/) is the way to go I think. Honestly, it feels like the way it's already gone, and it's just that the distros are fighting it tooth & nail whilst that approach takes over regardless.

kijin 5y ago
It's not just about disk space, though. Distros assume responsibility for fixing major bugs and security vulnerabilities in the packages they ship. Old versions often contain bugs and vulnerabilities that new versions don't. Distros have two choices here: either ship the new version and remove the old version, or backport the fix to the old version. Continuing to ship the old version without the fix is not an option -- even if you also ship the new version -- because some programs will inevitably use the old version and then the distro will be on the hook for any resulting hacks. Backporting every fix to every version that ever shipped is also not a realistic project. Here in the startup world we often forget that there's a whole other market where many people would gladly accept 3-year-old versions in exchange for a guarantee of security fixes for 5-10 years. Someone needs to cater to this market, and the (non-rolling) distros perform that thankless task because individual developers won't.
- scotty79 5y ago
  > Distros assume responsibility for fixing major bugs and security vulnerabilities in the packages they ship. I think they should just ship Python programs, not libraries. They could check what libraries given Python program uses are safe in the version that it uses them. And just don't care if each of Python programs has a separate copy of the libraries or if particular version of particular library is shared between Python programs by Python environment. Distributions might just give up responsibility for sharing Python packages between Python programs without giving up the responsibility for security of those programs.
  - kijin 5y ago
    So, patch two dozen copies of slightly different versions of the same library included in all those programs, instead of patching just one?
    - scotty79 5y ago
      You should patch two dozen python programs as a whole that use vulnerable version of libraries. Treat Python programs as if each of them was a single executable file and you only have information which versions of libraries it has inside it. And if any of those is known to be vulnerable treat whole program as a security threat.
    - thereddaikon 5y ago
      Why not? Its cheap resource wise whereas dependency hell is potentially debilitating. For some reason many proponents of the package management status quo are blind to this. Having multiple versions of a dependency is only bad in so far as its "messy". It isn't objectively bad. But having a system that breaks applications because two or more cant agree on a package version is objectively bad. Its arguing aesthetics versus getting the job done. A poor position. Windows, for all its faults, doesn't have this problem. It will happily accommodate multiple versions of say, .Net as needed.
      - kijin 5y ago
        Disk is cheap. RAM is cheap. Man-hours are not. Distros are maintained by people, who are often volunteers. You are asking them to do extra work (i.e. porting the same patch to multiple versions of the same library) so that someone else can have it easy. But why should they? Why not the other way around? It's not just aesthetics. If a new vulnerability is found in, say, libjpeg, then every Windows app that uses libjpeg needs to be updated separately. Tough luck if your favorite image manipulation tool takes a while to catch up. On the Linux side of the fence, the distro patches libjpeg and every app is automatically fixed. This is a huge win in terms of protecting the user. Why should we give up that benefit just because some developer wants to use his own special snowflake version of a library?
selfhoster11 5y ago
> Disk is plentiful, What makes you think so? SSDs aren't exactly stellar in the cost-per-TB department, as will be the case with each new higher-performance storage technology. Plenty of people cannot afford the prices of new Western tech either, what about them?
- danielheath 5y ago
  Text (eg python source code) is really quite small. Far too small to count unless you're a distro specifically targeting low-end machines.
- silvestrov 5y ago
  > cost-per-TB There you have it: You measure in TB, not gigabytes, not megabytes. Python packages are megabytes.
  - josefx 5y ago
    Not if they pull in all of their dependencies, PyQt would have a complete copy of all Qt binaries and a complete chrome install because of course Qt includes a browser based html viewer. Python packages are gigabytes.
    - zamadatix 5y ago
      What distro is pulling PyQt as a dependency of Python? There is a difference between "dependency" and "every package which has the word python in the description".
    - mccorrinall 5y ago
      PyQt only contains the bindings. You share the same Qt environment across your system (hence qmake needs to be in your path). The python package itself is not that big (~10 MB).
      - josefx 5y ago
        The comment on top of this chain is about letting every package specify its own versioned dependencies. So how would that global version work out when python needs 5.1 and some other software specifies 5.2?
goodpoint 5y ago
> I think this requirement made sense when disk space was scarce. No, the main reason is security. I need a distro to guarantee me that the library that I use are going to stay the same for the next 3 to 5 years, while also receiving small targeted patches. > I think this requirement makes sense if you trust that your distro is always better at choosing the 'best' version of a dependency that some software should use than the software author. No, it just has to be better at choosing version than picking them randomly using pip. Furthermore, when thousands of developers use the same combination of libraries from a distro the stack receives a ton of testing. The same cannot be said when each developer pick a random set of versions.
josefx 5y ago
> a la pnpm's approach for JS: https://pnpm.io/ https://pnpm.io/ Ah JS, how many days has it been since the last weekly "compromised npm package infecting everything" problem? If you are upholding that as gold standard you have to be the worlds laziest black hat. > Disk is plentiful, I recently had to install a chrome snap because it is the new IE6 and everyone is all over chrome exclusive APIs as if they were the new ActiveX. Over a gigabyte of dependencies for one application and given the trend of browser based desktop applications? I would like to have space left for my data after installing the programs I need for work.
cozzyd 5y ago
One case where disk space is somewhat scarce is on shared academic computing clusters (which often provide many versions of things via some module system, but your $HOME can have a quota that's just 30GB).

kijin 5y ago

It's not just about disk space, though. Distros assume responsibility for fixing major bugs and security vulnerabilities in the packages they ship. Old versions often contain bugs and vulnerabilities that new versions don't. Distros have two choices here: either ship the new version and remove the old version, or backport the fix to the old version. Continuing to ship the old version without the fix is not an option -- even if you also ship the new version -- because some programs will inevitably use the old version and then the distro will be on the hook for any resulting hacks. Backporting every fix to every version that ever shipped is also not a realistic project. Here in the startup world we often forget that there's a whole other market where many people would gladly accept 3-year-old versions in exchange for a guarantee of security fixes for 5-10 years. Someone needs to cater to this market, and the (non-rolling) distros perform that thankless task because individual developers won't.

scotty79 5y ago
> Distros assume responsibility for fixing major bugs and security vulnerabilities in the packages they ship. I think they should just ship Python programs, not libraries. They could check what libraries given Python program uses are safe in the version that it uses them. And just don't care if each of Python programs has a separate copy of the libraries or if particular version of particular library is shared between Python programs by Python environment. Distributions might just give up responsibility for sharing Python packages between Python programs without giving up the responsibility for security of those programs.
- kijin 5y ago
  So, patch two dozen copies of slightly different versions of the same library included in all those programs, instead of patching just one?
  - scotty79 5y ago
    You should patch two dozen python programs as a whole that use vulnerable version of libraries. Treat Python programs as if each of them was a single executable file and you only have information which versions of libraries it has inside it. And if any of those is known to be vulnerable treat whole program as a security threat.
  - thereddaikon 5y ago
    Why not? Its cheap resource wise whereas dependency hell is potentially debilitating. For some reason many proponents of the package management status quo are blind to this. Having multiple versions of a dependency is only bad in so far as its "messy". It isn't objectively bad. But having a system that breaks applications because two or more cant agree on a package version is objectively bad. Its arguing aesthetics versus getting the job done. A poor position. Windows, for all its faults, doesn't have this problem. It will happily accommodate multiple versions of say, .Net as needed.
    - kijin 5y ago
      Disk is cheap. RAM is cheap. Man-hours are not. Distros are maintained by people, who are often volunteers. You are asking them to do extra work (i.e. porting the same patch to multiple versions of the same library) so that someone else can have it easy. But why should they? Why not the other way around? It's not just aesthetics. If a new vulnerability is found in, say, libjpeg, then every Windows app that uses libjpeg needs to be updated separately. Tough luck if your favorite image manipulation tool takes a while to catch up. On the Linux side of the fence, the distro patches libjpeg and every app is automatically fixed. This is a huge win in terms of protecting the user. Why should we give up that benefit just because some developer wants to use his own special snowflake version of a library?

selfhoster11 5y ago

> Disk is plentiful, What makes you think so? SSDs aren't exactly stellar in the cost-per-TB department, as will be the case with each new higher-performance storage technology. Plenty of people cannot afford the prices of new Western tech either, what about them?

danielheath 5y ago
Text (eg python source code) is really quite small. Far too small to count unless you're a distro specifically targeting low-end machines.
silvestrov 5y ago
> cost-per-TB There you have it: You measure in TB, not gigabytes, not megabytes. Python packages are megabytes.
- josefx 5y ago
  Not if they pull in all of their dependencies, PyQt would have a complete copy of all Qt binaries and a complete chrome install because of course Qt includes a browser based html viewer. Python packages are gigabytes.
  - zamadatix 5y ago
    What distro is pulling PyQt as a dependency of Python? There is a difference between "dependency" and "every package which has the word python in the description".
  - mccorrinall 5y ago
    PyQt only contains the bindings. You share the same Qt environment across your system (hence qmake needs to be in your path). The python package itself is not that big (~10 MB).
    - josefx 5y ago
      The comment on top of this chain is about letting every package specify its own versioned dependencies. So how would that global version work out when python needs 5.1 and some other software specifies 5.2?

goodpoint 5y ago

> I think this requirement made sense when disk space was scarce. No, the main reason is security. I need a distro to guarantee me that the library that I use are going to stay the same for the next 3 to 5 years, while also receiving small targeted patches. > I think this requirement makes sense if you trust that your distro is always better at choosing the 'best' version of a dependency that some software should use than the software author. No, it just has to be better at choosing version than picking them randomly using pip. Furthermore, when thousands of developers use the same combination of libraries from a distro the stack receives a ton of testing. The same cannot be said when each developer pick a random set of versions.

josefx 5y ago

> a la pnpm's approach for JS: https://pnpm.io/ https://pnpm.io/ Ah JS, how many days has it been since the last weekly "compromised npm package infecting everything" problem? If you are upholding that as gold standard you have to be the worlds laziest black hat. > Disk is plentiful, I recently had to install a chrome snap because it is the new IE6 and everyone is all over chrome exclusive APIs as if they were the new ActiveX. Over a gigabyte of dependencies for one application and given the trend of browser based desktop applications? I would like to have space left for my data after installing the programs I need for work.

cozzyd 5y ago

One case where disk space is somewhat scarce is on shared academic computing clusters (which often provide many versions of things via some module system, but your $HOME can have a quota that's just 30GB).