4 ms·

> I’ve personally been burned on Python by the last bullet point where one of the dependencies required version 3.14 or less of module such and such, and anothe

> I’ve personally been burned on Python by the last bullet point where one of the dependencies required version 3.14 or less of module such and such, and another dependency needed 3.15 or higher. This is what people used to call “DLL Hell,” in Microsoft Windows. COM was supposed to fix that, but I don’t think it worked especially well.

NoZZz 2y ago
It required the re-implementation of stable interfaces as the components updated in version number; not everyone did that. I guess. A case of good plan, lazy execution.
a1o 2y ago
What I found mostly curious is I didn't even knew phyton 3.15 had already released
- lambda 2y ago
  The example was talking about versions of another package that both of the given packages depend on. Not Python itself. The version numbers were made up.
db48x 2y ago
Rust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size of your binaries, but you can end up with as many different versions of the common dependencies as you need.
- ChrisMarshallNY 2y ago
  The issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system. In practice, a lot of orgs just stopped including deprecated versions.
  - db48x 2y ago
    Yea, that was a mistake. Rust handles it transparently; if the program you write happens to depend on two versions of something, it just downloads them both for you.
    - ChrisMarshallNY 2y ago
      How does this work with linking? I would assume that both versions publish the same API, so does Rust add a mangling to the link reference? One of the things that made DLL Hell so bad, was that you would have no idea that you were calling a deprecated/new version of the function, until it started misbehaving.
      - PartiallyTyped 2y ago
        There is no dynamic linking in rust when it comes to other rust packages.
      - db48x 2y ago
        Rust does name mangling, and it basically just adds the crate version to the mix when it mangles the names. So if crates A and B both depend on X but different version, then A can only call functions from its chosen version of X while B can only call functions from its version. There is an optimization pass that discards duplicate functions, so functions in X that haven’t actually changed between versions will be deduplicated. This all happens automatically, so nobody ever has to think about it. Not the crate authors, and not you. What does get complicated is when the crates A and B both deal in types from X. If a function from A returns a type from X and a function from B takes that same type as an argument, then the compiler will step in with an error that the types don’t match. It’ll tell you that type Foo (from crate X version 1.0) doesn’t match Foo (from crate X version 2.0). This prevents all the possible runtime errors that could occur if you were really mixing an matching between both version. In that situation you will likely need to constrain your chosen versions of A and/or B such that they can agree on a single version of X, instead of allowing cargo to simply pick the latest available version.
- Animats 2y ago
  That only works if the dependency doesn't expose types used by multiple other dependencies. I keep running into this with the Rust 3D graphics stack. That's winit (windowing), wgpu (wrapper for Vulkan, Metal, WebGPU, etc.), egui (2D menu overlay), and support crates such as wgpu-egui, wgpu-profiling, tracy-profiler, and glam (vector and matrix math). Every time one of these has a breaking change, it takes 4 to 6 weeks before the whole stack works without patching. Then I have to fix three more levels of my own.
  - db48x 2y ago
    Yep. In principle you can examine the crates before you use them and pick ones that won’t ever expose you to that pitfall. In practice, of course, things are not always so ideal. But at least if you end up with incompatible versions then the compiler will step in and prevent your program from compiling, instead of allowing it to crash mysteriously later on. It’s frustrating, but not half so frustrating as it could be.
    - Animats 2y ago
      Yes. Cargo and the Rust compiler do catch clashes. A big advantage of that is that you seldom have to do a clean build in Rust. Make-based systems seem to require regular "make clean" cleanups, but Cargo has enough smarts to really know when it can avoid recompiling.
mardifoufs 2y ago
DLL hell hasn't been a thing for more than a decade at this point though, so maybe it did. I genuinely never encounter any issues with DLLs anymore.
- maccard 2y ago
  An awful lot of it went away in my experience in the c++11 world when everyone all Of a sudden wanted to upgrade. Then another time in 2015 when MSVC decided their standard library would be backwards compatible
- bregma 2y ago
  Most Windows apps ship with copies of all their dependencies. The DLL Hell problem was solved by disk and bandwidth getting cheaper to the point where many dozens or even hundreds of copies of MSCVRT.DLL and friends on your system pass unnoticed.
  - mardifoufs 2y ago
    But isn't that a good thing? I don't mind having more DLLs if stuff just works. It avoids the similar, though harder to solve issues on Linux. I'd rather be able to have a copy of the MSCVRT.DLL than to have to pray that my system has the right glibc to be honest. But maybe I'm not understanding the trade offs correctly.
    - cesarb 2y ago
      > I don't mind having more DLLs if stuff just works. The counterpart is when a remotely exploitable security issue is found in that DLL, as has happened with zlib in the past and log4j more recently; then you have to chase down and update every single copy of that DLL.
  - int_19h 2y ago
    If you absolutely need a way to install dependencies globally such that they can be shared between apps, you can still do that without version conflicts if you write proper manifests for them and use https://en.wikipedia.org/wiki/Side-by-side_assembly https://en.wikipedia.org/wiki/Side-by-side_assembly. But it's easier to just package a local copy and not worry about any of this, so that's what people mostly do now.
pjmlp 2y ago
COM has nothing to do with fixing DLL hell, it is as old as OLE, and uses DLLs as well. COM fixes having a cross language OOP ABI, that is all. The things that were supposed to fix that were: 1- Version resources, which hardly matter, because the app loading them has to validate the versions itself 2- .NET, hence the Global Assembly Cache and having the version as part of the Assembly (DLL) lookup. Still has issues if the lookup rules were badly configured on the app. 3- Application manifests, bringing the .NET ideas to Win32 4- Registry Free COM, extension of app manifests, allowing direct lookup of desired COM libraries without going through the registry for the version 5- UWP sandbox, didn't took off, only what is inside the sandbox is searched for 6- Easiest one, don't put stuff all over the place, only search inside an application specific directory
francisofascii 2y ago
Then later on NuGet was supposed to fix that, but even with a package manager you encounter "dll hell" like scenarios.
int_19h 2y ago
Not quite. DLL Hell was the situation when apps installed their dynamically linked dependencies globally, so that when two apps shipped different versions of the same library they could conflict. The situation was much rarer for dependencies because dependency trees simply weren't that deep in that era. It had also been solved since the introduction of assembly versioning over 20 years ago, which provided a standard way to have multiple versions of the same dependency installed and loaded side by side. Later Windows versions also added the ability to do the same for app-local DLLs, to the point where you can have different parts of your code get different versions of the DLL when they ask for it within the same process. So, for all practical purposes, it has been a solved problem for a very long time now.

NoZZz 2y ago

It required the re-implementation of stable interfaces as the components updated in version number; not everyone did that. I guess. A case of good plan, lazy execution.

a1o 2y ago

What I found mostly curious is I didn't even knew phyton 3.15 had already released

lambda 2y ago
The example was talking about versions of another package that both of the given packages depend on. Not Python itself. The version numbers were made up.

db48x 2y ago

Rust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size of your binaries, but you can end up with as many different versions of the common dependencies as you need.

ChrisMarshallNY 2y ago
The issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system. In practice, a lot of orgs just stopped including deprecated versions.
- db48x 2y ago
  Yea, that was a mistake. Rust handles it transparently; if the program you write happens to depend on two versions of something, it just downloads them both for you.
  - ChrisMarshallNY 2y ago
    How does this work with linking? I would assume that both versions publish the same API, so does Rust add a mangling to the link reference? One of the things that made DLL Hell so bad, was that you would have no idea that you were calling a deprecated/new version of the function, until it started misbehaving.
    - PartiallyTyped 2y ago
      There is no dynamic linking in rust when it comes to other rust packages.
    - db48x 2y ago
      Rust does name mangling, and it basically just adds the crate version to the mix when it mangles the names. So if crates A and B both depend on X but different version, then A can only call functions from its chosen version of X while B can only call functions from its version. There is an optimization pass that discards duplicate functions, so functions in X that haven’t actually changed between versions will be deduplicated. This all happens automatically, so nobody ever has to think about it. Not the crate authors, and not you. What does get complicated is when the crates A and B both deal in types from X. If a function from A returns a type from X and a function from B takes that same type as an argument, then the compiler will step in with an error that the types don’t match. It’ll tell you that type Foo (from crate X version 1.0) doesn’t match Foo (from crate X version 2.0). This prevents all the possible runtime errors that could occur if you were really mixing an matching between both version. In that situation you will likely need to constrain your chosen versions of A and/or B such that they can agree on a single version of X, instead of allowing cargo to simply pick the latest available version.
Animats 2y ago
That only works if the dependency doesn't expose types used by multiple other dependencies. I keep running into this with the Rust 3D graphics stack. That's winit (windowing), wgpu (wrapper for Vulkan, Metal, WebGPU, etc.), egui (2D menu overlay), and support crates such as wgpu-egui, wgpu-profiling, tracy-profiler, and glam (vector and matrix math). Every time one of these has a breaking change, it takes 4 to 6 weeks before the whole stack works without patching. Then I have to fix three more levels of my own.
- db48x 2y ago
  Yep. In principle you can examine the crates before you use them and pick ones that won’t ever expose you to that pitfall. In practice, of course, things are not always so ideal. But at least if you end up with incompatible versions then the compiler will step in and prevent your program from compiling, instead of allowing it to crash mysteriously later on. It’s frustrating, but not half so frustrating as it could be.
  - Animats 2y ago
    Yes. Cargo and the Rust compiler do catch clashes. A big advantage of that is that you seldom have to do a clean build in Rust. Make-based systems seem to require regular "make clean" cleanups, but Cargo has enough smarts to really know when it can avoid recompiling.

mardifoufs 2y ago

DLL hell hasn't been a thing for more than a decade at this point though, so maybe it did. I genuinely never encounter any issues with DLLs anymore.

maccard 2y ago
An awful lot of it went away in my experience in the c++11 world when everyone all Of a sudden wanted to upgrade. Then another time in 2015 when MSVC decided their standard library would be backwards compatible
bregma 2y ago
Most Windows apps ship with copies of all their dependencies. The DLL Hell problem was solved by disk and bandwidth getting cheaper to the point where many dozens or even hundreds of copies of MSCVRT.DLL and friends on your system pass unnoticed.
- mardifoufs 2y ago
  But isn't that a good thing? I don't mind having more DLLs if stuff just works. It avoids the similar, though harder to solve issues on Linux. I'd rather be able to have a copy of the MSCVRT.DLL than to have to pray that my system has the right glibc to be honest. But maybe I'm not understanding the trade offs correctly.
  - cesarb 2y ago
    > I don't mind having more DLLs if stuff just works. The counterpart is when a remotely exploitable security issue is found in that DLL, as has happened with zlib in the past and log4j more recently; then you have to chase down and update every single copy of that DLL.
- int_19h 2y ago
  If you absolutely need a way to install dependencies globally such that they can be shared between apps, you can still do that without version conflicts if you write proper manifests for them and use https://en.wikipedia.org/wiki/Side-by-side_assembly https://en.wikipedia.org/wiki/Side-by-side_assembly. But it's easier to just package a local copy and not worry about any of this, so that's what people mostly do now.

pjmlp 2y ago

COM has nothing to do with fixing DLL hell, it is as old as OLE, and uses DLLs as well. COM fixes having a cross language OOP ABI, that is all. The things that were supposed to fix that were: 1- Version resources, which hardly matter, because the app loading them has to validate the versions itself 2- .NET, hence the Global Assembly Cache and having the version as part of the Assembly (DLL) lookup. Still has issues if the lookup rules were badly configured on the app. 3- Application manifests, bringing the .NET ideas to Win32 4- Registry Free COM, extension of app manifests, allowing direct lookup of desired COM libraries without going through the registry for the version 5- UWP sandbox, didn't took off, only what is inside the sandbox is searched for 6- Easiest one, don't put stuff all over the place, only search inside an application specific directory

francisofascii 2y ago

Then later on NuGet was supposed to fix that, but even with a package manager you encounter "dll hell" like scenarios.

int_19h 2y ago

Not quite. DLL Hell was the situation when apps installed their dynamically linked dependencies globally, so that when two apps shipped different versions of the same library they could conflict. The situation was much rarer for dependencies because dependency trees simply weren't that deep in that era. It had also been solved since the introduction of assembly versioning over 20 years ago, which provided a standard way to have multiple versions of the same dependency installed and loaded side by side. Later Windows versions also added the ability to do the same for app-local DLLs, to the point where you can have different parts of your code get different versions of the DLL when they ask for it within the same process. So, for all practical purposes, it has been a solved problem for a very long time now.