5 ms·

To be honest, I am actually glad about this. Dynamic linking is the cause of a lot of complexity. First there is DLL hell on Windows and its equivalent on Linux

To be honest, I am actually glad about this. Dynamic linking is the cause of a lot of complexity. First there is DLL hell on Windows and its equivalent on Linux. Second, having to maintain a stable ABI is a huge drain on language evolution. In C++, there have been many features that have been vetoed by the compiler/standard library implementers because it would cause an ABI break. Because of this ABI concerns, C++ is stuck with overheads for unique_ptr, map, unordered_map, and regex classes that are inefficient. Secondly, a lot of the benefits of dynamic linking don’t make as much sense now. First with regards to space savings. With modern disk capacity having extra copies of executable code is probably not that big of a deal. In addition with Link Time Optimization, you actually may not be saving space with dynamic linking since the linker can pretty aggressively throw away code that it knows is ever called. Second, with regards to security. One thing to consider is that dynamic linking is so complex that it is one of the reasons why people use docker, to make sure their binary and its dependencies are stable. Once you have something in a docker, it has a lot of the same update issues as a static linked binary, only less efficient. In addition, with security, Rust unlike C and C++ is a memory safe language. Given the vast majority of security issues are memory safety issues, I would expect Rust to have a lot fewer CVE’s. I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily. Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily. By avoiding dynamic linking, I think, Rust positions itself best as the no-compromise, high performance, modern, systems programming language.

MR4D 6y ago
> has a lot of the same update issues as a static linked binary, only less efficient. I think this is an insightful comment. Obviously there are other reasons to use docker, but too often (cough...Python...cough) I see virtualization used for exactly that.
kaishin 6y ago
Swift has unfortunately no choice but to solve this problem if it is to be used to build native software for Apple platforms.
Animats 6y ago
I tend to agree. Dynamic linking is only a win on space when 1) you have multiple programs running which share the same library, 2) they're not all identical programs which would share code, and 3) the programs use a significant fraction of what's in each dynamic library they load. With static linking, you get to prune at the function level at link time. DLLs can't do that.
- coliveira 6y ago
  A major use of dynamic linking is loading extensions. Without DL you'll have a hard time using languages such as Python, or creating extensions to applications.
  - sgtnoodle 6y ago
    Dynamic linking and dynamic loading aren't necessarily the same thing, are they? I imagine one could statically link in some C code to a rust program that uses dlopen, for example, or even just do the syscall directly in rust (assuming that's possible).
    - pjmlp 6y ago
      Yes they are, exactly the same OS infrastructure. It doesn't matter if it is the kernel or your application that calls dlopen(). How do imagine that something like ld.so gets implemented?
      - sgtnoodle 6y ago
        The difference, I think, is that dynamic linking necessarily binds local symbols to the dynamically loaded foreign code, while dynamic loading simply loads the foreign code into the same address space and provides an API for looking up symbols' addresses by name. There's no inherent need for the ABI to match, as long as you don't directly try to call into a function; you can write adapters to explicitly handle the dynamically loaded code's ABI, like python does with ctypes.
randomNumber7 6y ago
> By avoiding dynamic linking, I think, Rust positions itself best as the no-compromise, high performance, modern, systems programming language. You can use static linking in c++ if you want to. There is 0 advantage in not having the option to use dynamic linking if you want to. It might be even more secure if e.g. the program links dynamically against a commonly used library and this library is updated regularly.
- deleted 6y ago
  [deleted]
- RcouF1uZ4gsC 6y ago
  > You can use static linking in c++ if you want to. There is 0 advantage in not having the option to use dynamic linking if you want to. While you can do static linking in C++ if you want to, you still pay the cost for support for dynamic linking because of ABI issues. You can static link, but you still lost out on new language features and changes that break ABI and so you are still stuck with sub-optimal standard library code. For example, for years, until GCC 5, GCC std lib had a non-standard, slower copy on write implementation of std::string, that stuck around for so long because it would break ABI to change it. Even now, in MSVC STL, there are a lot of performance optimizations which the maintainer acknowledges, but will have to wait because they would break ABI. So even if you just statically link and continuously build in C++, you are still paying the price for support of dynamic linking.
  - pjmlp 6y ago
    Microsoft breaks the ABI all the time, this is why there is usually several MSVCrt.dll versions installed. Visual Studio 2017 - 2019 were the first time they ever bothered with it, plus it is planned that VS vNext will break it again. This not counting the other C++ compilers on Windows with their own standard libraries, like C++ Builder and Intel C++. There is always COM/WinRT if I want an OOP ABI that doesn't break as easily.
- a1369209993 6y ago
  > There is 0 advantage in not having the option to use dynamic linking if you want to. The advantage is in other people (eg libc maintainers) not having the option to require you to use dynamic linking when you don't want to.
coliveira 6y ago
First of all, C/C++ can do static linking as well as dynamic linking. If static linking is preferred, people will do it. Second, dynamic linking is necessary because of how modern OSs are designed. Most GUIs are gigantic libraries that cannot be statically linked. Similarly for network code and other areas of modern OSs.
millstone 6y ago
Rust is certainly made simpler by not sweating ABI stability. But the main advantage of dynamic linking is to enable yesterday's app to run on tomorrow's OS (and vice-versa). It's why your apps keep working when you update your phone. Without a dynamic linking story, Rust is just not viable for writing UIKit, Android's frameworks, etc. Which is OK, it's a reasonable choice, but it limits Rust's scope.
- dralley 6y ago
  Zig is in a really good position here. Its featureset sticks closer to C, and being able to automatically generate C headers from Zig code and automatically consume C headers from Zig code is really convenient.
  - pjmlp 6y ago
    Not until it sorts out its use-after-free story.
  - millstone 6y ago
    Does Zig really solve this problem? For example, if I add a field to a Zig struct, can existing code still use the struct without needing to be recompiled?
    - dralley 6y ago
      No, but what I'm saying is, it makes dynamic linking a lot easier in general than Rust does.
ChrisMarshallNY 6y ago
If I use SPM, I seldom do dylibs. Since I wrote almost every package I consume, and the ones I didn't write are quite small; that works fine, for me.
kortex 6y ago
I largely agree and personally hate working with dynamic linking, but ultimately it depends on your architecture model - to share memory, or not to share memory. If you've already decided you don't want to share memory, and you don't want to use system IPC, that leaves you with sockets (network/unix/fifo), which are all just byte streams. If you have byte streams, then you gotta use (de)serialization (serde) on either end, so effectively there is one ABI: {uint8[], size_t}. This is the microservices/docker/cloud model, and it's great. For many purposes. But if you are dealing with OS level stuff, talking directly to hardware, in the embedded space, etc, now you care about different ABIs for performance reason. Now you decide, do I want static linking or dynamic linking? The only place I see dynamic linking really being a feature is when you have to link against the kernel, drivers, or proprietary subsystems (e.g. CUDA). IMHO, most userspace application layer stuff shouldn't be sharing memory, it should be communicating, again using byte streams and serde. So that basically leaves the hardware/software interface: kernel APIs, drivers, etc. If you're in this space, great, it makes sense to use dynamic linking with very stable ABIs with no generic shenanigans, you don't need them. I think a lot of pain of DLL hell comes from using dynamic libs when you really should be statically compiling if it's a library, and IPC if you need to communicate.
pjmlp 6y ago
>. One thing to consider is that dynamic linking is so complex that it is one of the reasons why people use docker, to make sure their binary and its dependencies are stable. Coding for almost 40 years and I am yet to use Docker to sort out this kind of issues. > I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily. When I started programming, dynamic linking was only available on big iron machines filling computer rooms, we did not need Go for knowing what static linking entails. > Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily. If Rust wants to succeed in replacing C and C++ in typical big corp, cargo better support binary libraries eventually.
- gpm 6y ago
  You must work in very different big corps than I, I have yet to see a binary .so we linked to but didn't have the source to.
  - pjmlp 6y ago
    The kind of corps where Oracle and SQL Server licenses are tiny drops on project expenses, do Windows, macOS, iOS, Android and UNIX in general. I am quite sure WebSphere, just to give an example, isn't releasing the source code for the .so that come along its Java implementation.
  - jcelerier 6y ago
    you live in a very shelled, blessed world (just looking at the fact that you said ".so", and not ".dll built with Visual Studio 2012").

MR4D 6y ago

> has a lot of the same update issues as a static linked binary, only less efficient. I think this is an insightful comment. Obviously there are other reasons to use docker, but too often (cough...Python...cough) I see virtualization used for exactly that.

kaishin 6y ago

Swift has unfortunately no choice but to solve this problem if it is to be used to build native software for Apple platforms.

Animats 6y ago

I tend to agree. Dynamic linking is only a win on space when 1) you have multiple programs running which share the same library, 2) they're not all identical programs which would share code, and 3) the programs use a significant fraction of what's in each dynamic library they load. With static linking, you get to prune at the function level at link time. DLLs can't do that.

coliveira 6y ago
A major use of dynamic linking is loading extensions. Without DL you'll have a hard time using languages such as Python, or creating extensions to applications.
- sgtnoodle 6y ago
  Dynamic linking and dynamic loading aren't necessarily the same thing, are they? I imagine one could statically link in some C code to a rust program that uses dlopen, for example, or even just do the syscall directly in rust (assuming that's possible).
  - pjmlp 6y ago
    Yes they are, exactly the same OS infrastructure. It doesn't matter if it is the kernel or your application that calls dlopen(). How do imagine that something like ld.so gets implemented?
    - sgtnoodle 6y ago
      The difference, I think, is that dynamic linking necessarily binds local symbols to the dynamically loaded foreign code, while dynamic loading simply loads the foreign code into the same address space and provides an API for looking up symbols' addresses by name. There's no inherent need for the ABI to match, as long as you don't directly try to call into a function; you can write adapters to explicitly handle the dynamically loaded code's ABI, like python does with ctypes.

randomNumber7 6y ago

> By avoiding dynamic linking, I think, Rust positions itself best as the no-compromise, high performance, modern, systems programming language. You can use static linking in c++ if you want to. There is 0 advantage in not having the option to use dynamic linking if you want to. It might be even more secure if e.g. the program links dynamically against a commonly used library and this library is updated regularly.

deleted 6y ago
[deleted]
RcouF1uZ4gsC 6y ago
> You can use static linking in c++ if you want to. There is 0 advantage in not having the option to use dynamic linking if you want to. While you can do static linking in C++ if you want to, you still pay the cost for support for dynamic linking because of ABI issues. You can static link, but you still lost out on new language features and changes that break ABI and so you are still stuck with sub-optimal standard library code. For example, for years, until GCC 5, GCC std lib had a non-standard, slower copy on write implementation of std::string, that stuck around for so long because it would break ABI to change it. Even now, in MSVC STL, there are a lot of performance optimizations which the maintainer acknowledges, but will have to wait because they would break ABI. So even if you just statically link and continuously build in C++, you are still paying the price for support of dynamic linking.
- pjmlp 6y ago
  Microsoft breaks the ABI all the time, this is why there is usually several MSVCrt.dll versions installed. Visual Studio 2017 - 2019 were the first time they ever bothered with it, plus it is planned that VS vNext will break it again. This not counting the other C++ compilers on Windows with their own standard libraries, like C++ Builder and Intel C++. There is always COM/WinRT if I want an OOP ABI that doesn't break as easily.
a1369209993 6y ago
> There is 0 advantage in not having the option to use dynamic linking if you want to. The advantage is in other people (eg libc maintainers) not having the option to require you to use dynamic linking when you don't want to.

coliveira 6y ago

First of all, C/C++ can do static linking as well as dynamic linking. If static linking is preferred, people will do it. Second, dynamic linking is necessary because of how modern OSs are designed. Most GUIs are gigantic libraries that cannot be statically linked. Similarly for network code and other areas of modern OSs.

millstone 6y ago

Rust is certainly made simpler by not sweating ABI stability. But the main advantage of dynamic linking is to enable yesterday's app to run on tomorrow's OS (and vice-versa). It's why your apps keep working when you update your phone. Without a dynamic linking story, Rust is just not viable for writing UIKit, Android's frameworks, etc. Which is OK, it's a reasonable choice, but it limits Rust's scope.

dralley 6y ago
Zig is in a really good position here. Its featureset sticks closer to C, and being able to automatically generate C headers from Zig code and automatically consume C headers from Zig code is really convenient.
- pjmlp 6y ago
  Not until it sorts out its use-after-free story.
- millstone 6y ago
  Does Zig really solve this problem? For example, if I add a field to a Zig struct, can existing code still use the struct without needing to be recompiled?
  - dralley 6y ago
    No, but what I'm saying is, it makes dynamic linking a lot easier in general than Rust does.

ChrisMarshallNY 6y ago

If I use SPM, I seldom do dylibs. Since I wrote almost every package I consume, and the ones I didn't write are quite small; that works fine, for me.

kortex 6y ago

I largely agree and personally hate working with dynamic linking, but ultimately it depends on your architecture model - to share memory, or not to share memory. If you've already decided you don't want to share memory, and you don't want to use system IPC, that leaves you with sockets (network/unix/fifo), which are all just byte streams. If you have byte streams, then you gotta use (de)serialization (serde) on either end, so effectively there is one ABI: {uint8[], size_t}. This is the microservices/docker/cloud model, and it's great. For many purposes. But if you are dealing with OS level stuff, talking directly to hardware, in the embedded space, etc, now you care about different ABIs for performance reason. Now you decide, do I want static linking or dynamic linking? The only place I see dynamic linking really being a feature is when you have to link against the kernel, drivers, or proprietary subsystems (e.g. CUDA). IMHO, most userspace application layer stuff shouldn't be sharing memory, it should be communicating, again using byte streams and serde. So that basically leaves the hardware/software interface: kernel APIs, drivers, etc. If you're in this space, great, it makes sense to use dynamic linking with very stable ABIs with no generic shenanigans, you don't need them. I think a lot of pain of DLL hell comes from using dynamic libs when you really should be statically compiling if it's a library, and IPC if you need to communicate.

pjmlp 6y ago

>. One thing to consider is that dynamic linking is so complex that it is one of the reasons why people use docker, to make sure their binary and its dependencies are stable. Coding for almost 40 years and I am yet to use Docker to sort out this kind of issues. > I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily. When I started programming, dynamic linking was only available on big iron machines filling computer rooms, we did not need Go for knowing what static linking entails. > Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily. If Rust wants to succeed in replacing C and C++ in typical big corp, cargo better support binary libraries eventually.

gpm 6y ago
You must work in very different big corps than I, I have yet to see a binary .so we linked to but didn't have the source to.
- pjmlp 6y ago
  The kind of corps where Oracle and SQL Server licenses are tiny drops on project expenses, do Windows, macOS, iOS, Android and UNIX in general. I am quite sure WebSphere, just to give an example, isn't releasing the source code for the .so that come along its Java implementation.
- jcelerier 6y ago
  you live in a very shelled, blessed world (just looking at the fact that you said ".so", and not ".dll built with Visual Studio 2012").