4 ms·

Normally you end up only a single version of a library needed in the scope of one application. E.g. application A depends on libraries B and C and both of these

by ambrop7 10y ago

Normally you end up only a single version of a library needed in the scope of one application. E.g. application A depends on libraries B and C and both of these depend on the same version of library D. And there is no problem as long as this is the case, the exact same library will be used. If for some reason B and C depend on different versions of a library, then things may go wrong, e.g. one package may end up inadvertently using an incorrect version of a dependency. Nix cannot solve this problem because it originates from the OS/compiler design (same sonames, same symbol names). Consider that with respect to libraries, Nix mostly just wraps the compiler to add needed -L flags and also sets RPATH in shared libs and executables so that needed shared libs are found at runtime. If an application effectively depends on different versions of one library with the same soname, the dynamic linker will still use just one of the available ones. What Nix does allow you to do is to isolate packages from one another. You can have one application using only one version of library D, and another application using only a different version of library D. I should note that it's somehow hard to get to this situation, because the Nix packages collection (nixpkgs) doesn't keep too many versions of packages. Typically for minor upgrades the library is just bumped, and when you rebuild the final application all references to that library will be up to date. You can only realistically get issues if the application ends up depending on different major versions which are both maintained in nixpkgs but are not designed to work together in the same application.

nine_k 10y ago
I suspect that the same binary cannot realistically depend on two different versions of a library. I can imagine that package A may have two binaries, one depending on a library B version 1.1, and another, on B 1.2 Does NixOS allow to have both B-1.1 and B-1.2 installed, and A's different binaries to load the correct different versions of the library from B-1.1 and B-1.2?
- ambrop7 10y ago
  Well yes strictly speaking, the same binary. But in practice with Nix's default library resolution logic (the mentioned compiler wrapping) you will probably also have problems trying to build binaries in one package using version of a library (see note [1]). That is, I think you'd have to do some "hacking" to fix the issue, or separate the binaries into different packages (and maybe create a new package that "assembles" all the binaries by creating symlinks to them). > Does NixOS allow to have both B-1.1 and B-1.2 installed, and A's different binaries to load the correct different versions of the library from B-1.1 and B-1.2? Yes, well actually, the thing is that typically you only "install" (into a user environment) final applications (A) that you need not libraries, so the libraries (B) can be considered to not really be installed, just available (somewhere in /nix/store) and referenced correctly (e.g. by RPATH). [1] When you declare a dependency in a package specification by putting it into buildInputs, the compiler wrapper will automatically add an -L<store_path>/lib so that linking with just -l will find the library. With multiple versions of a library in buildInputs, that becomes a problem.
- dbcurtis 10y ago
  As I understand it, Nixos runs their own patched versions of ld and the dynamic loading system calls in libc. The normal lib.so search mechanism is replaced wholesale with "thou shalt link against this dll and no other" on a per-execuatble basis. Multiple versions of .so's don't cause problems, because no executable is depending on a search mechanism it doesn't control.
  - ambrop7 10y ago
    I think that they don't do too drastic changes. There are certainly checks in the compilation process to not accidentally link to /usr/lib/ libraries, but I think the main problem of finding libraries works with a simple compiler wrapper adding needed -Lflags (for finding libs at compile time) and RPATH entries to binaries (for finding libs at run time). There of course may be "hacks" made to gcc/libc but I don't see a reason they would be essential for functioning. One notable thing Nix does is hardcode a path to a specific dynamic linker (e.g. /nix/store/HASH-glibc-2.23/lib/ld-linux-x86-64.so.2). All other dynamic libraries should be referenced through soname+RPATH.
    - dbcurtis 10y ago
      I agree that the changes are likely not extremely invasive, and I would hesitate to call them "hacks". OTOH, it does make Nixos fundamentally different from other (u)nix systems. They are basically saying it's time to hard-fork the (u)nix philosophy about how dynamic linking is done. They might be right. It's a very interesting approach, and I want to understand it in detail. I haven't had the time to bring up a Nixos system yet, though.
  - vertex-four 10y ago
    Actually, I think they use the standard ld.so... it's just that for every executable, the RPATH in the header of the executable is set to a list of directories where the libraries it was built against can be found. ld.so uses the RPATH to find libraries by default.
    - ambrop7 10y ago

by ambrop7 10y ago

nine_k 10y ago

I suspect that the same binary cannot realistically depend on two different versions of a library. I can imagine that package A may have two binaries, one depending on a library B version 1.1, and another, on B 1.2 Does NixOS allow to have both B-1.1 and B-1.2 installed, and A's different binaries to load the correct different versions of the library from B-1.1 and B-1.2?

ambrop7 10y ago
Well yes strictly speaking, the same binary. But in practice with Nix's default library resolution logic (the mentioned compiler wrapping) you will probably also have problems trying to build binaries in one package using version of a library (see note [1]). That is, I think you'd have to do some "hacking" to fix the issue, or separate the binaries into different packages (and maybe create a new package that "assembles" all the binaries by creating symlinks to them). > Does NixOS allow to have both B-1.1 and B-1.2 installed, and A's different binaries to load the correct different versions of the library from B-1.1 and B-1.2? Yes, well actually, the thing is that typically you only "install" (into a user environment) final applications (A) that you need not libraries, so the libraries (B) can be considered to not really be installed, just available (somewhere in /nix/store) and referenced correctly (e.g. by RPATH). [1] When you declare a dependency in a package specification by putting it into buildInputs, the compiler wrapper will automatically add an -L<store_path>/lib so that linking with just -l will find the library. With multiple versions of a library in buildInputs, that becomes a problem.
dbcurtis 10y ago
As I understand it, Nixos runs their own patched versions of ld and the dynamic loading system calls in libc. The normal lib.so search mechanism is replaced wholesale with "thou shalt link against this dll and no other" on a per-execuatble basis. Multiple versions of .so's don't cause problems, because no executable is depending on a search mechanism it doesn't control.
- ambrop7 10y ago
  I think that they don't do too drastic changes. There are certainly checks in the compilation process to not accidentally link to /usr/lib/ libraries, but I think the main problem of finding libraries works with a simple compiler wrapper adding needed -Lflags (for finding libs at compile time) and RPATH entries to binaries (for finding libs at run time). There of course may be "hacks" made to gcc/libc but I don't see a reason they would be essential for functioning. One notable thing Nix does is hardcode a path to a specific dynamic linker (e.g. /nix/store/HASH-glibc-2.23/lib/ld-linux-x86-64.so.2). All other dynamic libraries should be referenced through soname+RPATH.
  - dbcurtis 10y ago
    I agree that the changes are likely not extremely invasive, and I would hesitate to call them "hacks". OTOH, it does make Nixos fundamentally different from other (u)nix systems. They are basically saying it's time to hard-fork the (u)nix philosophy about how dynamic linking is done. They might be right. It's a very interesting approach, and I want to understand it in detail. I haven't had the time to bring up a Nixos system yet, though.
- vertex-four 10y ago
  Actually, I think they use the standard ld.so... it's just that for every executable, the RPATH in the header of the executable is set to a list of directories where the libraries it was built against can be found. ld.so uses the RPATH to find libraries by default.
  - ambrop7 10y ago