3 ms·

Rust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size

by db48x 2y ago

Rust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size of your binaries, but you can end up with as many different versions of the common dependencies as you need.

ChrisMarshallNY 2y ago
The issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system. In practice, a lot of orgs just stopped including deprecated versions.
- db48x 2y ago
  Yea, that was a mistake. Rust handles it transparently; if the program you write happens to depend on two versions of something, it just downloads them both for you.
  - ChrisMarshallNY 2y ago
    How does this work with linking? I would assume that both versions publish the same API, so does Rust add a mangling to the link reference? One of the things that made DLL Hell so bad, was that you would have no idea that you were calling a deprecated/new version of the function, until it started misbehaving.
    - PartiallyTyped 2y ago
      There is no dynamic linking in rust when it comes to other rust packages.
    - db48x 2y ago
      Rust does name mangling, and it basically just adds the crate version to the mix when it mangles the names. So if crates A and B both depend on X but different version, then A can only call functions from its chosen version of X while B can only call functions from its version. There is an optimization pass that discards duplicate functions, so functions in X that haven’t actually changed between versions will be deduplicated. This all happens automatically, so nobody ever has to think about it. Not the crate authors, and not you. What does get complicated is when the crates A and B both deal in types from X. If a function from A returns a type from X and a function from B takes that same type as an argument, then the compiler will step in with an error that the types don’t match. It’ll tell you that type Foo (from crate X version 1.0) doesn’t match Foo (from crate X version 2.0). This prevents all the possible runtime errors that could occur if you were really mixing an matching between both version. In that situation you will likely need to constrain your chosen versions of A and/or B such that they can agree on a single version of X, instead of allowing cargo to simply pick the latest available version.
      - norir 2y ago
        I'm sure that cargo does a clever job of all of this but this kind of functionality is precisely why I find rust so off putting. It encourages you to take on huge amounts of unnecessary complexity (including complicated dependency trees) and then tries to hide that complexity in abstraction. But in practice these are always leaky abstractions that _someone_ (and likely you) will have to pay for. At a baseline, the poor compilation times and byzantine of rust are to me the most obvious symptoms of this embrace of complexity.
        db48x 2y ago
        If the language you use don’t have any way of dealing with this problem then one day you will be dealing with heap corruption because a library you (transitively) depend on changed the size of a struct. This is the fate of every C and C++ programmer. Or you could use a safe dynamic language like Lisp, Javascript, Python, and many others. Programmers using these languages are so productive because they never have to put up with any nonsense like random heap corruption. Or you could use Rust, which gives you the safety without any of the runtime costs incurred by the dynamic languages. What you call Byzantine complexity is in fact a very simple rule that you can teach to anybody, and which in practice most developers never need to deal with. It doesn’t even slow down the compiler. Engineering is all about tradeoffs, so choose wisely.
Animats 2y ago
That only works if the dependency doesn't expose types used by multiple other dependencies. I keep running into this with the Rust 3D graphics stack. That's winit (windowing), wgpu (wrapper for Vulkan, Metal, WebGPU, etc.), egui (2D menu overlay), and support crates such as wgpu-egui, wgpu-profiling, tracy-profiler, and glam (vector and matrix math). Every time one of these has a breaking change, it takes 4 to 6 weeks before the whole stack works without patching. Then I have to fix three more levels of my own.
- db48x 2y ago
  Yep. In principle you can examine the crates before you use them and pick ones that won’t ever expose you to that pitfall. In practice, of course, things are not always so ideal. But at least if you end up with incompatible versions then the compiler will step in and prevent your program from compiling, instead of allowing it to crash mysteriously later on. It’s frustrating, but not half so frustrating as it could be.
  - Animats 2y ago
    Yes. Cargo and the Rust compiler do catch clashes. A big advantage of that is that you seldom have to do a clean build in Rust. Make-based systems seem to require regular "make clean" cleanups, but Cargo has enough smarts to really know when it can avoid recompiling.

by db48x 2y ago

ChrisMarshallNY 2y ago

The issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system. In practice, a lot of orgs just stopped including deprecated versions.

db48x 2y ago
Yea, that was a mistake. Rust handles it transparently; if the program you write happens to depend on two versions of something, it just downloads them both for you.
- ChrisMarshallNY 2y ago
  How does this work with linking? I would assume that both versions publish the same API, so does Rust add a mangling to the link reference? One of the things that made DLL Hell so bad, was that you would have no idea that you were calling a deprecated/new version of the function, until it started misbehaving.
  - PartiallyTyped 2y ago
    There is no dynamic linking in rust when it comes to other rust packages.
  - db48x 2y ago
    Rust does name mangling, and it basically just adds the crate version to the mix when it mangles the names. So if crates A and B both depend on X but different version, then A can only call functions from its chosen version of X while B can only call functions from its version. There is an optimization pass that discards duplicate functions, so functions in X that haven’t actually changed between versions will be deduplicated. This all happens automatically, so nobody ever has to think about it. Not the crate authors, and not you. What does get complicated is when the crates A and B both deal in types from X. If a function from A returns a type from X and a function from B takes that same type as an argument, then the compiler will step in with an error that the types don’t match. It’ll tell you that type Foo (from crate X version 1.0) doesn’t match Foo (from crate X version 2.0). This prevents all the possible runtime errors that could occur if you were really mixing an matching between both version. In that situation you will likely need to constrain your chosen versions of A and/or B such that they can agree on a single version of X, instead of allowing cargo to simply pick the latest available version.
    - norir 2y ago
      I'm sure that cargo does a clever job of all of this but this kind of functionality is precisely why I find rust so off putting. It encourages you to take on huge amounts of unnecessary complexity (including complicated dependency trees) and then tries to hide that complexity in abstraction. But in practice these are always leaky abstractions that _someone_ (and likely you) will have to pay for. At a baseline, the poor compilation times and byzantine of rust are to me the most obvious symptoms of this embrace of complexity.
      - db48x 2y ago
        If the language you use don’t have any way of dealing with this problem then one day you will be dealing with heap corruption because a library you (transitively) depend on changed the size of a struct. This is the fate of every C and C++ programmer. Or you could use a safe dynamic language like Lisp, Javascript, Python, and many others. Programmers using these languages are so productive because they never have to put up with any nonsense like random heap corruption. Or you could use Rust, which gives you the safety without any of the runtime costs incurred by the dynamic languages. What you call Byzantine complexity is in fact a very simple rule that you can teach to anybody, and which in practice most developers never need to deal with. It doesn’t even slow down the compiler. Engineering is all about tradeoffs, so choose wisely.

Animats 2y ago

That only works if the dependency doesn't expose types used by multiple other dependencies. I keep running into this with the Rust 3D graphics stack. That's winit (windowing), wgpu (wrapper for Vulkan, Metal, WebGPU, etc.), egui (2D menu overlay), and support crates such as wgpu-egui, wgpu-profiling, tracy-profiler, and glam (vector and matrix math). Every time one of these has a breaking change, it takes 4 to 6 weeks before the whole stack works without patching. Then I have to fix three more levels of my own.

db48x 2y ago
Yep. In principle you can examine the crates before you use them and pick ones that won’t ever expose you to that pitfall. In practice, of course, things are not always so ideal. But at least if you end up with incompatible versions then the compiler will step in and prevent your program from compiling, instead of allowing it to crash mysteriously later on. It’s frustrating, but not half so frustrating as it could be.
- Animats 2y ago
  Yes. Cargo and the Rust compiler do catch clashes. A big advantage of that is that you seldom have to do a clean build in Rust. Make-based systems seem to require regular "make clean" cleanups, but Cargo has enough smarts to really know when it can avoid recompiling.