5 ms·

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a singl

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compile every_single_time because apps could reference binaries. The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice. More often than not, when I upgrade a library, I find myself having to upgrade my app’s code because so much has changed. The burden of having to constantly backup, upgrade, manually tweak config files, over and over and over again for days/weeks/months was SO not worth the few hundred megabytes or whatever dynamic loading was supposed to have saved.

witten 12y ago
> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. This is how NixOS works: http://nixos.org/ http://nixos.org/
deathanatos 12y ago
> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? Also, a program A depends on library B and library C v1.1. Library B also depends on C, but v1.2. Which gets used? > More often than not, when I upgrade a library, I find myself having to upgrade my app’s code because so much has changed. To me, this is the point of major version numbers. If you break clients, you increment the major version number, resulting in libfoo.so.2 and libfoo.so.3. Then, the scheme becomes much like hashes, in that newer versions won't break older clients, except you get security patches and a single copy of the library. However, the responsibility of knowing when to increment the major is left to a human, and all the error that entails. As a sibling notes, there are distros out there that do this. (They are not my preference, for the above reasons.)
- noobermin 12y ago
  I'm no expert, but based on the parent, I think it comes down to what he means by "library hash." If it's a hash (like a checksum) of the whole shared library, then different versions would have different hashes, theoretically.
- zackmorris 12y ago
  Ya it would basically ignore names/versions and just use the hash as the identifier, a bit like BitTorrent. Unfortunately you do bring up a good point that security patches generally won't work without a recompile of the parent binary. One possible way out of this is that the external interface/unit tests for a library could also have their own hash, so if a library fixes something like a buffer exploit without changing its interface, the parent binary could use the new drop-in replacement. In practice I’m skeptical if this would work reliably though, because binaries may be relying on idiosyncrasies. I'm thinking that a simpler method would be to have patches use the hash system. So say curl uses libssl and libssl releases a security update, then someone could drop the new libssl into the curl project and rebuild it without having to touch any code, giving curl a new hash that could be installed by other users. I think we are used to this almost never working so we are hesitant to upgrade. But the idea would be that we’d upgrade binaries that depend on libraries (rather than just libraries) and it would be a really cheap operation compared to today.
- chongli 12y ago
  >Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? No, this is the responsibility of the server which distributes binaries to users via the package manager.
  - palunon 12y ago
    User, as in programmer who uses the library.
    - chongli 12y ago
      Programmers are used to recompiling all the time. What's the problem here?
coherentpony 12y ago
> The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice. It works in practice if developers and maintainers adhere to semantic versioning. Unfortunately, there are numerous packages that don't adhere to this standard and that's when widespread breakage occurs.
- vidarh 12y ago
  An example of where this has worked (though admittedly the reason it has continued to work as long as it has is that the size of the community has dwindled down to a very small, manageable number over the last 20 years) is AmigaOS, where there are 30 year old libraries that are still updated, and where the updates are still expected to be drop-in replacements for the previous version.
- zokier 12y ago
  And if developers would just write bug-free code then also life would be much simpler. Sadly neither of these things is going to happen, so we need to deal with it somehow.
deleted 12y ago
[deleted]

witten 12y ago

> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. This is how NixOS works: http://nixos.org/ http://nixos.org/

deathanatos 12y ago

> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? Also, a program A depends on library B and library C v1.1. Library B also depends on C, but v1.2. Which gets used? > More often than not, when I upgrade a library, I find myself having to upgrade my app’s code because so much has changed. To me, this is the point of major version numbers. If you break clients, you increment the major version number, resulting in libfoo.so.2 and libfoo.so.3. Then, the scheme becomes much like hashes, in that newer versions won't break older clients, except you get security patches and a single copy of the library. However, the responsibility of knowing when to increment the major is left to a human, and all the error that entails. As a sibling notes, there are distros out there that do this. (They are not my preference, for the above reasons.)

noobermin 12y ago
I'm no expert, but based on the parent, I think it comes down to what he means by "library hash." If it's a hash (like a checksum) of the whole shared library, then different versions would have different hashes, theoretically.
zackmorris 12y ago
Ya it would basically ignore names/versions and just use the hash as the identifier, a bit like BitTorrent. Unfortunately you do bring up a good point that security patches generally won't work without a recompile of the parent binary. One possible way out of this is that the external interface/unit tests for a library could also have their own hash, so if a library fixes something like a buffer exploit without changing its interface, the parent binary could use the new drop-in replacement. In practice I’m skeptical if this would work reliably though, because binaries may be relying on idiosyncrasies. I'm thinking that a simpler method would be to have patches use the hash system. So say curl uses libssl and libssl releases a security update, then someone could drop the new libssl into the curl project and rebuild it without having to touch any code, giving curl a new hash that could be installed by other users. I think we are used to this almost never working so we are hesitant to upgrade. But the idea would be that we’d upgrade binaries that depend on libraries (rather than just libraries) and it would be a really cheap operation compared to today.
chongli 12y ago
>Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? No, this is the responsibility of the server which distributes binaries to users via the package manager.
- palunon 12y ago
  User, as in programmer who uses the library.
  - chongli 12y ago
    Programmers are used to recompiling all the time. What's the problem here?

coherentpony 12y ago

> The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice. It works in practice if developers and maintainers adhere to semantic versioning. Unfortunately, there are numerous packages that don't adhere to this standard and that's when widespread breakage occurs.

vidarh 12y ago
An example of where this has worked (though admittedly the reason it has continued to work as long as it has is that the size of the community has dwindled down to a very small, manageable number over the last 20 years) is AmigaOS, where there are 30 year old libraries that are still updated, and where the updates are still expected to be drop-in replacements for the previous version.
zokier 12y ago
And if developers would just write bug-free code then also life would be much simpler. Sadly neither of these things is going to happen, so we need to deal with it somehow.

deleted 12y ago

[deleted]