6 ms·

I’ve been convinced static linking is the way to go for a number of years. The benefits of dynamic linking don’t make sense anymore imo

by klysm 3y ago

intelVISA 3y ago
Awoken
tssge 3y ago
On single user systems, sure. However multi user systems can benefit greatly from dynamic linking; some highly multi user systems become impossible to create at all with only static linking (RAM demand becomes too great). Dynamic linking also leads to better CPU cache utilization and thus higher performance; provided if multiple applications are actually utilizing the dynamically linked library. While RAM is somewhat abundant these days, L1/L2/L3 cache on your CPU most definitely isn't. Application startup times are also of higher performance when part of the code is already in memory. EDIT: Let me elaborate a bit on what I mean by "benefiting greatly" with a real world example. I have a system with around 300 tenants with a minimum of 500 PHP processes online at any given time. A single PHP process is able to load 70MB worth of dynamically linked libraries (PHP extensions) if so chosen by the tenant. I'll just skip libc and other such libraries to give static linking a chance. With dynamic linking, the maximum memory requirement is 70MB for said extensions. With static linking, the maximum memory requirement is 35GB for said extensions. Dynamic linking thus leads to 99,8% cost saving on memory usage. Bear in mind this is at minimum load: static linking requirements will scale linearly with load. Not to even get started on the performance benefits this brings...
- gpm 3y ago
  Are these extensions standard across your users, or are they user supplied? If the former is the case, in a world where dynamic linking wasn't "the way things were done", you'd just just statically link them all into the PHP interpreters binary, and then only run the code to actually start them for the users that want them. Linux shares memory between the different instances of the same executable the same way it does between different instances of a shared library, there should be no difference in cost. Today, I doubt php supports this, particularly the "only run the code to actually start them" part, but in principle it's a lot simpler than dynamic linking to do that. If the latter is the case, how are you getting any savings today?
  - tssge 3y ago
    >Are these extensions standard across your users, or are they user supplied? These extensions are standard PHP extensions, so they are the same for each process and user. Extensions like MySQL driver, Postgres driver, ImageMagick bindings and such. You are correct that no savings would be possible if these were different across each user.
    - a1369209993 3y ago
      > These extensions are standard PHP extensions, so they are the same for each process and user. Then with static linking, the maximum memory requirement is 70MB for said extensions (statically linked into the PHP executable).
      - tssge 3y ago
        Are you basically saying there is no difference in memory usage when comparing static linking and dynamic linking? Any links to more info on the matter? I tested this on the machine with a static executable I compiled it looks to me that memory is not shared when statically linked. I thought the point of dynamic linking anyways was to save memory by sharing it (and disk space, though these days we have more than enough for executable code). Though it could be just that I misunderstand the output of smaps_rollup.
        rcxdude 3y ago
        Multiple executions of the same executable will share the code pages, so assuming all the users are executing the same binary on disk, the memory should be shared. Shared libraries only give you additional deduplication in the case that multiple binaries are loading the same shared libraries.
        rfoo 3y ago
        > Are you basically saying there is no difference in memory usage when comparing static linking and dynamic linking? In your case, yes, there is (almost [1]) no difference at all. If you have 500 different PHP binaries (say, different version of PHP) with same set of extension `.so`-s shared between them (is this even possible?? but let's assume it is), then your example works. [1] Almost, because if you also linked libc statically (not recommended as glibc is specifically designed to be hostile to static linking), you can't share libc with other processes (e.g. nginx, systemd, etc) so you need ~2MB more memory, this is still paid only once, not multiplied by how many php binary you run, though.
- cryptonector 3y ago
  When the changes to Solaris 10 got pushed that removed static link archives for the OS/Net core, and which replaced all the statically-linked /bin executables with dynamically-linked versions, boot times improved drastically owing to less I/O being needed because of the sharing.
josephg 3y ago
I go back and forth on this. On MacOS, I do like that the UI libraries for all applications are shipped with the OS. When the OS updates, the UI for all installed programs updates to include the new look and feel. Apple can also update them to change the way that applications render to the screen. These libraries are quite large, and they probably shouldn't be shipped with every application. If they were, it would make it a lot harder to compile an app to work with multiple versions of macos. Apps would probably break a lot more when the OS updates. For shell scripts and such, I agree and I'd be happier if everything was statically linked. My biggest criticism of dynamic linking is how ridiculous the situation is on debian and similar OSes. Imagine if I make some software with rust (or javascript or python or something). My software might pull in 20-100 dependencies from cargo, npm, pip, etc. If I want my software to appear in apt, the debian maintainers insist on adding a mirror of all of those dependencies into apt. So apt sort of maintains a crappy, out of date mirror of a bunch of other language-specific package managers. And as far as I know, they do this work by hand - adding misery to misery. Nobody wants that. Least of all the software developers - who will get bug reports for their own software compiled with mysteriously out-of-date dependencies that they aren't testing themselves. Its horrible.
- AnonymousPlanet 3y ago
  I don't understand what you think pip, npm etc. have to do with dynamic linking. Those libraries get loaded in the respective interpreter, the linker doesn't come into action at all. And security wise npm, pip and the likes are nightmares waiting to be exploited xz style. No static linking will save you from it.
  - josephg 2y ago
    It maps. If you throw a nodejs or python package in apt, you can either include all of its js / python dependencies in the package itself or make separate apt packages for each of the projects dependencies and have the apt package depend on those packages. The difference is pretty similar to static / dynamic linking. Including the dependencies gives the package author explicit control over the versions of all your dependencies. But depending on external apt packages means you can install security updates to shared libraries system wide. (Think log4j or xz). And as I understand it, that is what Debian prefers. In JavaScript projects it’s even become common practice to use a bundler to “compile” server side JavaScript (with all your dependencies) into a single large unreadable .js file. Doing that can reduce memory usage and dramatically improve startup time. The bundled code usually has dead code elimination. It’s more or less identical to static linking - except the resulting artifact is another JavaScript file.

I’ve been convinced static linking is the way to go for a number of years. The benefits of dynamic linking don’t make sense anymore imo

by klysm 3y ago

I’ve been convinced static linking is the way to go for a number of years. The benefits of dynamic linking don’t make sense anymore imo

intelVISA 3y ago

Awoken

tssge 3y ago

On single user systems, sure. However multi user systems can benefit greatly from dynamic linking; some highly multi user systems become impossible to create at all with only static linking (RAM demand becomes too great). Dynamic linking also leads to better CPU cache utilization and thus higher performance; provided if multiple applications are actually utilizing the dynamically linked library. While RAM is somewhat abundant these days, L1/L2/L3 cache on your CPU most definitely isn't. Application startup times are also of higher performance when part of the code is already in memory. EDIT: Let me elaborate a bit on what I mean by "benefiting greatly" with a real world example. I have a system with around 300 tenants with a minimum of 500 PHP processes online at any given time. A single PHP process is able to load 70MB worth of dynamically linked libraries (PHP extensions) if so chosen by the tenant. I'll just skip libc and other such libraries to give static linking a chance. With dynamic linking, the maximum memory requirement is 70MB for said extensions. With static linking, the maximum memory requirement is 35GB for said extensions. Dynamic linking thus leads to 99,8% cost saving on memory usage. Bear in mind this is at minimum load: static linking requirements will scale linearly with load. Not to even get started on the performance benefits this brings...

gpm 3y ago
Are these extensions standard across your users, or are they user supplied? If the former is the case, in a world where dynamic linking wasn't "the way things were done", you'd just just statically link them all into the PHP interpreters binary, and then only run the code to actually start them for the users that want them. Linux shares memory between the different instances of the same executable the same way it does between different instances of a shared library, there should be no difference in cost. Today, I doubt php supports this, particularly the "only run the code to actually start them" part, but in principle it's a lot simpler than dynamic linking to do that. If the latter is the case, how are you getting any savings today?
- tssge 3y ago
  >Are these extensions standard across your users, or are they user supplied? These extensions are standard PHP extensions, so they are the same for each process and user. Extensions like MySQL driver, Postgres driver, ImageMagick bindings and such. You are correct that no savings would be possible if these were different across each user.
  - a1369209993 3y ago
    > These extensions are standard PHP extensions, so they are the same for each process and user. Then with static linking, the maximum memory requirement is 70MB for said extensions (statically linked into the PHP executable).
    - tssge 3y ago
      Are you basically saying there is no difference in memory usage when comparing static linking and dynamic linking? Any links to more info on the matter? I tested this on the machine with a static executable I compiled it looks to me that memory is not shared when statically linked. I thought the point of dynamic linking anyways was to save memory by sharing it (and disk space, though these days we have more than enough for executable code). Though it could be just that I misunderstand the output of smaps_rollup.
      - rcxdude 3y ago
        Multiple executions of the same executable will share the code pages, so assuming all the users are executing the same binary on disk, the memory should be shared. Shared libraries only give you additional deduplication in the case that multiple binaries are loading the same shared libraries.
      - rfoo 3y ago
        > Are you basically saying there is no difference in memory usage when comparing static linking and dynamic linking? In your case, yes, there is (almost [1]) no difference at all. If you have 500 different PHP binaries (say, different version of PHP) with same set of extension `.so`-s shared between them (is this even possible?? but let's assume it is), then your example works. [1] Almost, because if you also linked libc statically (not recommended as glibc is specifically designed to be hostile to static linking), you can't share libc with other processes (e.g. nginx, systemd, etc) so you need ~2MB more memory, this is still paid only once, not multiplied by how many php binary you run, though.
cryptonector 3y ago
When the changes to Solaris 10 got pushed that removed static link archives for the OS/Net core, and which replaced all the statically-linked /bin executables with dynamically-linked versions, boot times improved drastically owing to less I/O being needed because of the sharing.

josephg 3y ago

I go back and forth on this. On MacOS, I do like that the UI libraries for all applications are shipped with the OS. When the OS updates, the UI for all installed programs updates to include the new look and feel. Apple can also update them to change the way that applications render to the screen. These libraries are quite large, and they probably shouldn't be shipped with every application. If they were, it would make it a lot harder to compile an app to work with multiple versions of macos. Apps would probably break a lot more when the OS updates. For shell scripts and such, I agree and I'd be happier if everything was statically linked. My biggest criticism of dynamic linking is how ridiculous the situation is on debian and similar OSes. Imagine if I make some software with rust (or javascript or python or something). My software might pull in 20-100 dependencies from cargo, npm, pip, etc. If I want my software to appear in apt, the debian maintainers insist on adding a mirror of all of those dependencies into apt. So apt sort of maintains a crappy, out of date mirror of a bunch of other language-specific package managers. And as far as I know, they do this work by hand - adding misery to misery. Nobody wants that. Least of all the software developers - who will get bug reports for their own software compiled with mysteriously out-of-date dependencies that they aren't testing themselves. Its horrible.

AnonymousPlanet 3y ago
I don't understand what you think pip, npm etc. have to do with dynamic linking. Those libraries get loaded in the respective interpreter, the linker doesn't come into action at all. And security wise npm, pip and the likes are nightmares waiting to be exploited xz style. No static linking will save you from it.
- josephg 2y ago
  It maps. If you throw a nodejs or python package in apt, you can either include all of its js / python dependencies in the package itself or make separate apt packages for each of the projects dependencies and have the apt package depend on those packages. The difference is pretty similar to static / dynamic linking. Including the dependencies gives the package author explicit control over the versions of all your dependencies. But depending on external apt packages means you can install security updates to shared libraries system wide. (Think log4j or xz). And as I understand it, that is what Debian prefers. In JavaScript projects it’s even become common practice to use a bundler to “compile” server side JavaScript (with all your dependencies) into a single large unreadable .js file. Doing that can reduce memory usage and dramatically improve startup time. The bundled code usually has dead code elimination. It’s more or less identical to static linking - except the resulting artifact is another JavaScript file.