8 ms·

Language preferences and safety features aside, I for one am 100% convinced that a/the upcoming language to rule "systems programming" (in a FOSS world with com

by c0l0 5y ago

Language preferences and safety features aside, I for one am 100% convinced that a/the upcoming language to rule "systems programming" (in a FOSS world with community participation) MUST have strong support for dynamic linking. If you look at something like `apt-cache show podman | grep ^Built-Using:` (Output: https://paste.debian.net/plain/1225449 https://paste.debian.net/plain/1225449) on Debian 11, you will see why. Imagine a few of those components shared between tens of packages, and security problems discovered im some. It's got to be any package maintainer's worst nightmare.

dilyevsky 5y ago
How does having strong support for dynamic linking prevent someone from copypasting insecure code or just linking statically with your .so? Dynamic linking just makes deployment and compatibility a pain for some very dubious advantages
- gspr 5y ago
  > How does having strong support for dynamic linking prevent someone from copypasting insecure code or just linking statically with your .so? It doesn't. But it gives people a real alternative to doing what you describe! And only when a real alternative to something bad exists, one can reasonably demand that people stop doing the bad thing (Debian does, so does Fedora and a few other distros I believe).
choeger 5y ago
I agree. But I also think that the following two language features are important: 1. Polymorphism 2. Specialization of code and data structures Both aspects only go well together with global compilation which basically enforces static linking, unless you either substantially improve the ABI or introduce some artificial language limits regarding composition. Personally, I think that it should be possible to extend the C-ABI with some meaningful form of specialization: You effectively only need offsets and sizes in order to specialize code. But I don't have a prototype to explore this idea further.
- benbenolson 5y ago
  This is hilarious: someone comments about a real problem that requires a practical solution in a programming language, and someone invariably rushes to add their subset of “nice to have” language features.
  - naasking 5y ago
    Well motivated choices though. Polymorphism reduces lines of code, which are correlated with bug count. Reliability is essential for critical system software. Specialization eliminates nearly all overhead of polymorphism so there's no disincentive against using it.
gravypod 5y ago
> It's got to be any package maintainer's worst nightmare. Is this really a problem in most modern package management systems and build systems? I think Nix is a great example of handling this gracefully by just tracking the inputs and outputs of each compilation and sandboxing the compilation so we can guarantee the build definition includes all specified deps. This is something debian can build on top of apt.
- c0l0 5y ago
  I guess one can work around the lack of dynamic linking in a language like Go on the package manager and/or distro level (though some established distros and package management ecosystems, which I very much hope will not go away, will have it harder to adapt than others), but I really would like to make everyone think about whether we should... Imagine a Debian archive (that is, a few tens of thousands of packages with source, build recipies and binaries for multiple architectures available) with only static linking on the table. You got a few hundreds to thousands of those packages installed on typical, real-world Debian machines. Then, a central component like zlib or OpenSSL gets an important patch. This triggers hundreds, if not tens of thousands (transitive library .so dependencies are a thing after all, and you want to have all your supported arches covered) of rebuilds across the archive. All users will have to re-download and re-install all resulting upgraded packages they have installed. With increased sharing of source code (i.e., when following "don't build your own crypto!" and other generally useful mantras by re-using existing codebases) and only static linking on the table, this kind of problems gets really, really bad really quickly for large-ish software archives. Kinda like with log4j in the Java ecosystem, where each app is used and event expected to dragging all its runtime dependencies along on its own. In the dylib world we are living in right now however, users get a single updated package, and upon the next restart of an application linking against it, they're immunized against the bug. The reduction of churn is nigh-unfathomably massive. If your systems generally only run a single application, and there's a dedicated ops team at the top (or bottom - it's a matter of perspective, I guess :)) of a monorepo world with each dependency fully vendored in, and an "intelligent" build and deployment system atop it all that will take care of gracefully upgrading everything everywhere upon such routine patching, I can see how statically linking everything reduces the amount of potential pain points upon deploying stuff. For the significant portion that makes up the rest of the world, they generally don't have such luxuries, and will be a lot worse off. Especially established and wonderful Free Software communities like Debian or Fedora. I think that would be a huge sacrifice to make.

by c0l0 5y ago

dilyevsky 5y ago

How does having strong support for dynamic linking prevent someone from copypasting insecure code or just linking statically with your .so? Dynamic linking just makes deployment and compatibility a pain for some very dubious advantages

gspr 5y ago
> How does having strong support for dynamic linking prevent someone from copypasting insecure code or just linking statically with your .so? It doesn't. But it gives people a real alternative to doing what you describe! And only when a real alternative to something bad exists, one can reasonably demand that people stop doing the bad thing (Debian does, so does Fedora and a few other distros I believe).

choeger 5y ago

I agree. But I also think that the following two language features are important: 1. Polymorphism 2. Specialization of code and data structures Both aspects only go well together with global compilation which basically enforces static linking, unless you either substantially improve the ABI or introduce some artificial language limits regarding composition. Personally, I think that it should be possible to extend the C-ABI with some meaningful form of specialization: You effectively only need offsets and sizes in order to specialize code. But I don't have a prototype to explore this idea further.

benbenolson 5y ago
This is hilarious: someone comments about a real problem that requires a practical solution in a programming language, and someone invariably rushes to add their subset of “nice to have” language features.
- naasking 5y ago
  Well motivated choices though. Polymorphism reduces lines of code, which are correlated with bug count. Reliability is essential for critical system software. Specialization eliminates nearly all overhead of polymorphism so there's no disincentive against using it.

gravypod 5y ago

> It's got to be any package maintainer's worst nightmare. Is this really a problem in most modern package management systems and build systems? I think Nix is a great example of handling this gracefully by just tracking the inputs and outputs of each compilation and sandboxing the compilation so we can guarantee the build definition includes all specified deps. This is something debian can build on top of apt.

c0l0 5y ago
I guess one can work around the lack of dynamic linking in a language like Go on the package manager and/or distro level (though some established distros and package management ecosystems, which I very much hope will not go away, will have it harder to adapt than others), but I really would like to make everyone think about whether we should... Imagine a Debian archive (that is, a few tens of thousands of packages with source, build recipies and binaries for multiple architectures available) with only static linking on the table. You got a few hundreds to thousands of those packages installed on typical, real-world Debian machines. Then, a central component like zlib or OpenSSL gets an important patch. This triggers hundreds, if not tens of thousands (transitive library .so dependencies are a thing after all, and you want to have all your supported arches covered) of rebuilds across the archive. All users will have to re-download and re-install all resulting upgraded packages they have installed. With increased sharing of source code (i.e., when following "don't build your own crypto!" and other generally useful mantras by re-using existing codebases) and only static linking on the table, this kind of problems gets really, really bad really quickly for large-ish software archives. Kinda like with log4j in the Java ecosystem, where each app is used and event expected to dragging all its runtime dependencies along on its own. In the dylib world we are living in right now however, users get a single updated package, and upon the next restart of an application linking against it, they're immunized against the bug. The reduction of churn is nigh-unfathomably massive. If your systems generally only run a single application, and there's a dedicated ops team at the top (or bottom - it's a matter of perspective, I guess :)) of a monorepo world with each dependency fully vendored in, and an "intelligent" build and deployment system atop it all that will take care of gracefully upgrading everything everywhere upon such routine patching, I can see how statically linking everything reduces the amount of potential pain points upon deploying stuff. For the significant portion that makes up the rest of the world, they generally don't have such luxuries, and will be a lot worse off. Especially established and wonderful Free Software communities like Debian or Fedora. I think that would be a huge sacrifice to make.