3 ms·

> As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program agai

> As I understand it, Rust does support dynamic linking for pure rust code. It just doesn't have a stable ABI yet. Not quite. To safely run a Rust program against a Rust dynamic library, you had to have compiled and linked that program against exactly that version of the Rust library. This is necessary for fundamental Rust type system reasons. It’s not simply a matter of stabilizing ABI. It’s not like updating the library without relinking would work if you used exactly the same rustc version - it would still be unsafe. So no, Rust doesn’t support dynamic linking in any meaningful way. > And most linux programs don't make such heavy use of dynamic linking to work. This is not true at all. Linux relies on dynamic linking extensively in two ways: - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. - Lots of Linux programs rely on the dynamically loaded module architecture like what pam does. All of the scripting language runtimes do this for example.

gpm 11mo ago
The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.
- pizlonator 11mo ago
  > The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
  - gpm 11mo ago
    It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.
josephg 11mo ago
> Linux relies on dynamic linking extensively in two ways: > > - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. This isn't a linux thing. Its a popular linux distribution thing. Linux distributions want to be in charge of which version of which dependencies every package uses. But that means the same piece of software on different linux machines will use subtly different versions of all their dependencies. Distribution maintainers love this, because they can patch libjpeg to fix a bug and it'll affect all programs on the system. But its hell for the actual developers of those packages. If I use a dependency at version 1.2, ubuntu might silently replace that with version 1.1 or 1.2.1 or something else - none of which I've actually tested. Debian might only have version 0.9 available, so they quietly try and patch my software to use that instead. If there's a bug as a result, people will blame me for the bug. Sometimes the debian maintainers maintain their own patch sets to fix bugs in old versions - so they ship version 0.9.0-1 or something. When a user is having problems its now almost completely impossible for me to reproduce their environment on my computer. Linux doesn't have to do any of this. It all works great with static linked binaries. Eg alpine linux uses musl instead of glibc and just statically links everything. This provides better performance in many cases because library code can be inlined. Apt is in theory language agnostic. But in practice, this whole dynamic linking thing only really works for pure C libraries. Even C++ is a bit too fragile in most cases for this to work. (I think C++ is in theory ABI compatible, but in practice it only seems to work reliably if you compile all binaries with the same compiler, against the same version of the C++ STL.) The reason Go doesn't support dynamic linking is that Rob Pike thinks its stupid. That dynamic linking only ever made sense when we didn't have much RAM, and those days are way behind us. I think I generally agree with him. I have no idea where that leaves rust.
- pizlonator 11mo ago
  > This isn't a linux thing. Its a popular linux distribution thing You mean: it’s an OS thing. Windows relies on it. MacOS and iOS rely on it. When shared frameworks get to the multiple GB size then you can’t boot the system without dynamic linking. > Even C++ is a bit too fragile in most cases for this to work. (I think C++ is in theory ABI compatible, but in practice it only seems to work reliably if you compile all binaries with the same compiler, against the same version of the C++ STL.) Maybe about a quarter of the shared libraries, and maybe half of the large ones (as measured by source size) on Linux are C++. It’s true that the preferred idiom is to use C ABI around the edges, but not everyone does this. C++ has stable ABI even if you use different compilers these days. That’s been true for at least a decade now. > The reason Go doesn't support dynamic linking is that Rob Pike thinks its stupid I don’t care about what Rob Pike thinks is stupid. > That dynamic linking only ever made sense when we didn't have much RAM, and those days are way behind us Are you ok with me making thay same argument in favor of Fil-C’s current weak sauce optimization story? “Optimizing your language implementation only even made sense when we didn’t have much RAM and when computers were slow, and those days are way behind us.”

gpm 11mo ago

The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. However if rust wanted to gain this ability, it basically is just a matter of a stable ABI. The only other thing you would need is to get a hash of all the parts of the system that are equivalent to C header files and have the program check the hash in the dylib matched the version it was compiled against at load time. I believe in fact the sdylib crate type in nightly rust does (or at least is intended to do, not sure it's fully implemented) exactly this.

pizlonator 11mo ago
> The ability to replace libraries without recompiling downstream dependents is not a necessary component of dynamic linking. Yes it is, in the sense that not having this is a massive bug. Dynamic linking becomes horrifically unsafe without this property, rendering it mostly useless.
- gpm 11mo ago
  It's not a bug, it's just a lack of an additional feature on top of dynamic linking. In fact it prevents bugs as it provides a clear bar on the misuse of dynamic linking as is so common in C/C++/(as designed) Fil-C to replace libraries with likely-ABI-incompatible replacements. Fil-C limits the blast radius of bugs caused by this, but doesn't prevent the bugs. Just saying "you can't do that" like current rust actually prevents them. (Alternatively growing that feature properly, including checking ABI compatibility, as intended with sdylib's would also prevent them. To the best of my knowledge no C-like language does this today) Dynmaic linking used responsibly to reduce linking times in development cycles, and to reduce memory usage when you have memory things using the same library, remains perfectly safe without this feature. The former is reasonably common in the rust ecosystem today, and I've literally never hit or heard of anyone else hitting a safety issue as a result. Minimizing memory usage is a niche use case in the modern world... but it's there if you want it. You just have to not modify the application post compilation - just like you have to not modify any application post compilation to retain memory safety guarantees.

josephg 11mo ago

> Linux relies on dynamic linking extensively in two ways: > > - A dynamic library may be updated independently of its dependents. You can’t do this in Rust, see above. This isn't a linux thing. Its a popular linux distribution thing. Linux distributions want to be in charge of which version of which dependencies every package uses. But that means the same piece of software on different linux machines will use subtly different versions of all their dependencies. Distribution maintainers love this, because they can patch libjpeg to fix a bug and it'll affect all programs on the system. But its hell for the actual developers of those packages. If I use a dependency at version 1.2, ubuntu might silently replace that with version 1.1 or 1.2.1 or something else - none of which I've actually tested. Debian might only have version 0.9 available, so they quietly try and patch my software to use that instead. If there's a bug as a result, people will blame me for the bug. Sometimes the debian maintainers maintain their own patch sets to fix bugs in old versions - so they ship version 0.9.0-1 or something. When a user is having problems its now almost completely impossible for me to reproduce their environment on my computer. Linux doesn't have to do any of this. It all works great with static linked binaries. Eg alpine linux uses musl instead of glibc and just statically links everything. This provides better performance in many cases because library code can be inlined. Apt is in theory language agnostic. But in practice, this whole dynamic linking thing only really works for pure C libraries. Even C++ is a bit too fragile in most cases for this to work. (I think C++ is in theory ABI compatible, but in practice it only seems to work reliably if you compile all binaries with the same compiler, against the same version of the C++ STL.) The reason Go doesn't support dynamic linking is that Rob Pike thinks its stupid. That dynamic linking only ever made sense when we didn't have much RAM, and those days are way behind us. I think I generally agree with him. I have no idea where that leaves rust.

pizlonator 11mo ago
> This isn't a linux thing. Its a popular linux distribution thing You mean: it’s an OS thing. Windows relies on it. MacOS and iOS rely on it. When shared frameworks get to the multiple GB size then you can’t boot the system without dynamic linking. > Even C++ is a bit too fragile in most cases for this to work. (I think C++ is in theory ABI compatible, but in practice it only seems to work reliably if you compile all binaries with the same compiler, against the same version of the C++ STL.) Maybe about a quarter of the shared libraries, and maybe half of the large ones (as measured by source size) on Linux are C++. It’s true that the preferred idiom is to use C ABI around the edges, but not everyone does this. C++ has stable ABI even if you use different compilers these days. That’s been true for at least a decade now. > The reason Go doesn't support dynamic linking is that Rob Pike thinks its stupid I don’t care about what Rob Pike thinks is stupid. > That dynamic linking only ever made sense when we didn't have much RAM, and those days are way behind us Are you ok with me making thay same argument in favor of Fil-C’s current weak sauce optimization story? “Optimizing your language implementation only even made sense when we didn’t have much RAM and when computers were slow, and those days are way behind us.”