16 ms·

Static Linking Considered Harmful (2004)

39 points by vortex_ape 6y ago

rvz 6y ago
Given the movement to static linking in compiled languages like LLVM, Go, Rust for deploying self-contained servers and cross-platform single binary executables, I think this guy has clearly lost the argument. Both him and the authors of glibc.
- greatgib 6y ago
  I don't agree with you. I think that we are on the top of a tide of the usual 'old is new again' hype, but at some point people will realize again all the issues and limitations of that and go back to dynamic linking. In the same way as containers per application is trendy now. So far it we are encountering all the drawbacks and limitations that used to be known long time ago and that once leaded to dynamic loading...
  - rvz 6y ago
    > ...but at some point people will realize again all the issues and limitations of that and go back to dynamic linking. Except that for using third-party libraries and distributing apps or games, there is no reason to use dynamically linked libraries here and also in software distribution. I guess if you talked to iOS / Android app developers who complained about dynamically linked third-party libraries that slowed app execution startup times, desktop app developers who wanted a single statically-linked executable that prevents their app being hijacked with a malicious dynamically linked library via LD_PRELOAD or DYLD_INSERT_LIBRARIES or in general developers or users don't want to have to upgrade other dependencies using dynamically linked libraries to install your software, then I guess one can say it would be better to use static binaries here. Who's to say that a hacker can simply modify a .dll, .dylib or .so in your game and enable developer cheats to ruin it. The authors point on LD_PRELOAD alongside with 'security benefits' is nothing more than outdated and has now been intellectually bankrupted.
    - kvez 6y ago
      I am more interested in security for users than for developers. Sure, it's convenient and might be slightly more work for users to mess with your app, but I don't trust every proprietary app developer to be monitoring the CVEs for every library they use. I'm sure there's a ton of multiplayer games out there that statically linked things like libpng and networking libraries, and by now are an open door into the systems of anyone playing online.
      - jcelerier 6y ago
        > I'm sure there's a ton of multiplayer games out there that statically linked things like libpng and networking libraries you know that even when they are dynamically linked, the developer just ships the dlls next to the app right ?
    - saagarjha 6y ago
      > The authors point on LD_PRELOAD alongside with 'security benefits' is nothing more than outdated and has now been intellectually bankrupted. I LD_PRELOAD (and DYLD_INSERT_LIBRARIES) code into every process I spawn from the command line, except setuid executables (and sometimes even then–I jump through the hoops necessary to make this work), because it lets me extend compiled applications in arbitrary ways to better suit whatever I want them to do. Saying that my usecase is "outdated and…intellectually bankrupted" just because you're not aware of the very real and very convenient benefits of function interposing is the real ignorance here.
    - greatgib 6y ago
      Security is a wrong argument. You can have dynamic loading and locked source folders and disabled LD_PRELOAD. At the opposite, remember that each statically loaded exe would have to constantly be updated/recompiled for all the libraries updates that they use or could let a lot of security holes opened for a long time.
      - ghoward 6y ago
        On the flip side, with libraries that use version symbols, you need to update all of the versions, so dynamic linking might have some of the same problem.
w0utert 6y ago
Didn't see one of the IMO biggest downsides of static linking in that list, which is that there is no sane way to prevent symbol clashes if different components of your application need different versions of the same dependency. When using shared linking dynamic libraries can link some of their dependencies statically into the .so, using strictly local symbols, allowing the application to load it even if it already links some other version of the same dependency. We had this problem at my workplace where we developed a component that needed to embed a relatively modern version of the Lua interpreter. When the team that developed the application that used our component tried to link it, they found out that some ancient component they also had been linking in for some completely different and unrelated purpose had a transitive dependency on Lua 5.0, released almost 20 years ago. For backwards compatibility reasons it was not an option to modify the old component to use the newer Lua version, and linking both Lua versions statically into the same binary is impossible. This would never have been a problem if we could have shipped our component as a shared library (which we already build and use for other internal applications), because there the Lua interpreter could have been linked into the shared library privately.
- Joker_vD 6y ago
  Um, there is this thing called objcopy: --redefine-sym old=new Change the name of a symbol old, to new. This can be useful when one is trying link two things together for which you have no source, and there are name collisions. --redefine-syms=filename Apply --redefine-sym to each symbol pair "old new" listed in the file filename. filename is simply a flat file, with one symbol pair per line. Line comments may be introduced by the hash character. This option may be given more than once. So it's absolutely possible, just ugly.
  - w0utert 6y ago
    I’m not sure I understand how this would help? When linking the main application there will be two component .a files linked into it, that both contain references to symbols that should be resolved to something in a liblua.a, so that needs to be linked along as well, but one component needs to resolve them to liblua.a from Lua 5.2, the other to liblua.a from 5.0. Most (but not all) of the Lua symbols from both versions have the same names. How can objcopy resolve this? Multi-pass linking or something like that?
sesuximo 6y ago
What about: - LTO - small libraries/huge pages can link multiple libraries to one page - lower overhead on first call assuming you lazily link
- pjmlp 6y ago
  Solved by bytecode distribution formats that AOT/JIT compile, and yes there are those for C and C++ as well.
  - saagarjha 6y ago
    JIT compilation is not feasible on a number of platforms.
    - pjmlp 6y ago
      True, but AOT compilation at install time like on watchOS or mainframe language environments is.
  - sesuximo 6y ago
    Sorry if I’m being daft but can you provide a link? Something that will dynamically do LTO with shared libraries
    - pjmlp 6y ago
      GraalVM/Sulong and CLR devirtualization.
tyho 6y ago
None of this is really relevant when using containers though. The moment you containerise your application, you have effectively statically linked it. Surely a big reason containers became popular is because you can avoid all the massive headaches you get when deploying dynamically linked software.
- salawat 6y ago
  I don't understand people who like containerization. So now instead of just taking on the onus of maintaining a software library, now I'm going to have to fuck around with maintaining how I expect it to be deployed, and force other people to adhere to that model, thereby forcing my paradigms and alternate system architecture/exotic filesystem and concomitant weirdness onto the user? I don't know, it just never clicked for me. Seems like a solution that appeals to people who like spending most of their time in config/build files than actually using a system, but maybe I'm just a curmudgeon/haven't used it right.
  - judge2020 6y ago
    Docker is for applications. Using it in an actual library for a programming language should be limited to setting up the dev environment needed to run tests [for the benefit of consistency]. It makes sense for someone like Discourse to only support running via docker since they want to make sure you're running with the exact same dependencies they're running with which makes issue-reporting and debugging easier.
alanfranz 6y ago
Static Linking vs Dynamic Linking is a choice that has tradeoffs. If you only consider what's good about dynamic linking and ignore its shortcomings, sure: dynamic linking is The Right Choice! But I see no mention of those tradeoffs. Examples: what if the system-provided library is old or has bugs? What if you're delivering to multiple distributions (or even OSes) and you need a consistent behaviour? What if you need different versions of a library?
- baby 6y ago
  What if you need to instrument these libraries?
- cortesoft 6y ago
  Yeah, this article seems a bit disingenuous... it says "I will prove there are no benefits to static linking!", but then goes on just to mention positives about dynamic linking. It never even mentions the reasons why people choose static linking, let alone attempt to refute them.
- silvestrov 6y ago
  also: What if the 1.0.1 version of the library isn't 100% compatible with the 1.0.0 and the app now crashes? Semantic versioning is nice in theory, but you can't rely on everybody getting in 100% correct and not introducing obscure bugs due to slightly different memory layout of structs. It also makes testing apps more difficult as you now doesn't know exactly what code the clients are running as there can be an incredible high number of permutations of library versions. Dynamic linking also makes it impossible to do link-time optimizations (e.g. inlining, dead-strip, locating code near the function that calls it)
- rbarrois 6y ago
  I'd say that the deal is: - With Dynamic Linking, it's the job of the system admin (typically a distribution maintainer) to ensure that the system-provided library and application are compatible — and add patches if needed; - With Static Linking, the package developer takes control of the whole experience; they deem more important to deliver a working application to the end user than to ensure compatibility with an OS. This makes sense for isolated applications, e.g. video games. For a system's core stack (coretools, X11, etc.), the end user expects to receive security fixes in a timely manner. From the distribution's point of view, it seems simpler to have a single library to patch, recompile and push, than to have the maintainer of each package statically linked against said library release an updated version...
deleted 6y ago
[deleted]
SAI_Peregrinus 6y ago
Dynamic linking provides some benefits: You save disk space and some memory, especially for things that are used by most of the system like libc. You gain the ability to fix some problems in libraries without updating every app that uses the library. Dynamic linking as implemented by most UNIX-Like OSes has a huge drawback: you only have one system copy of any given library, so all your applications are stuck using the same copy. There's no easy way to have multiple versions (say, with different features) of the same library without giving them different names. This isn't really a failing of dynamic linking, it's a failing of the OS library management model. Dynamic linking in general has some other drawbacks: you lose Link-Time Optimization, there's more overhead at load time (and on the first call with lazy linking), and a few other performance-related issues can crop up on occasion. There's also the issue that replacing a dynamically linked library is a lot easier than replacing a statically linked one for an attacker, so dynamic libraries can allow for vulnerabilities to be introduced (though this also is an OS issue, and there are often good mitigations for it).
- arminiusreturns 6y ago
  I had always hoped the way Gobolinux approached this would start being adopted by other distros.
  - SAI_Peregrinus 6y ago
    Yeah, it seems ideal. But I've not used it, yet, mostly because it's such a big change.
  - Slackwise 6y ago
    I'm more interested in the Nix/Guix approach, where you get the benefit of exact version linking, but also an "immutable" version controlled OS, so things like Puppet aren't necessary to manage state. (And likewise, deploy small VMs/containers with just the package manager tools without needing Docker as the entire OS state can be rebuilt using a definition file/version hash.)
- dfox 6y ago
  Almost any ELF based dynamic linker has support not only for versioned library dependencies but also for versioned symbols in these libraries. The issue is mostly that some widely used libraries implement this wrong and to larger extent that people who try to ship linux binaries just link to whatever latest version they have on their build system.
brundolf 6y ago
The "X Considered Harmful" trope has become exhausting ("harmful", even?). Every single technical decision has trade-offs. You might make a case that you (usually) shouldn't do X in cases like Y, or maybe even that you (usually) shouldn't do X in general, but it's a very big statement to say that something is flatly harmful. Very, very few things are so simple. If you aren't Edsger Dijkstra, there's a good chance you're outing yourself as either inexperienced, or willfully ignorant, by claiming them to be so. Yet this meme has encouraged lots of people to do just that, by making dogmatism easy and catchy.
- judge2020 6y ago
  https://meyerweb.com/eric/comment/chech.html https://meyerweb.com/eric/comment/chech.html https://news.ycombinator.com/item?id=9744916 https://news.ycombinator.com/item?id=9744916
danShumway 6y ago
I wanted to quote a few parts of the article that I considered incorrect, but then realized I would be quoting most of the article. Short version, almost anyone who's ever tried to ship a nontrivial game on Linux should realize this article is wrong. I particularly disagree with the idea that doing any dynamic linking in a program at all means that instantly it's just as non-portable as a fully dynamically linked program. These things are a continuum, there are degrees of fragility -- you can make a program that breaks less often when ported to new systems. See some of Ethan Lee's writing on this topic.[0] On a meta-level, you should be instantly suspicious of any programming advice that includes quotes like: > Conclusion: Never use [thing]! > This has never been the case and never will be the case. In reality, most software techniques have benefits and downsides. If a debate has survived for over a decade, that means that the answer probably isn't so completely universal that you can completely ignore one side. I'm not going to say that there are no black-and-white correct answers in software development, but people who regularly claim to have found them should be regarded with suspicion. On average, they'll often just be oversimplifying things. [0]: https://gist.github.com/flibitijibibo/b67910842ab95bb3decdf89d1502de88 https://gist.github.com/flibitijibibo/b67910842ab95bb3decdf8...
- klodolph 6y ago
  Games on Linux are a beast, aren’t they! I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application. For example, it is fairly reasonable to require LibSDL2 to be installed. However, if you bundle LibSDL2 with your app, you’re now directly dependent on things like X11 and ALSA.
  - AnIdiotOnTheNet 6y ago
    > I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application. Herein lies the problem, because Linux has absolutely no distinction between "system" and "application". Everything is part of the system.
  - 6y ago
nkurz 6y ago
I was trying to figure out how old this page was, but I'm still not certain. The headers show a "Last-Modified" of 2017, but it's definitely much earlier than that. I think the main content dates from at least 2004 from when it was on Redhat's site: https://web.archive.org/web/20041221091001/http://people.redhat.com:80/drepper/no_static_linking.html https://web.archive.org/web/20041221091001/http://people.red... The Clarification looks to have been added in late 2006: https://web.archive.org/web/20061119024251/http://people.redhat.com/drepper/no_static_linking.html https://web.archive.org/web/20061119024251/http://people.red...
airocker 6y ago
Dynamic linking has a major versioning problem. We shipped a product that was an .so file. We once spent months chasing a bug in the field where some of the clients would load a previous version of libstdc++ , calls to our .so would crash the application because we were linked against a higher version. Most ABi is backward compatible but not forward compatible. Plus there is the benefit of being more portable.
ed25519FUUU 6y ago
> fixes (either security or only bug) have to be applied to only one place: the new DSO(s). Yes, and all of your bugs can be introduced in a new single place as well! Seriously though. People love static linking because it solves more problems than it introduces, despite its downsides. I will argue that it’s easier to keep a system secure with static linking. You can upgrade the service that is insecure immediately. Or else you need to wait for a Big Bang deployment, and it can be held up by some random thing that isn’t even in the critical path.
maple3142 6y ago
Although I get the point of dynamic linking (security, size...), I still prefer CLI to be written in a language that enable you to easilly build a static binary such as Go, Rust. IMO, the benefit of static binary is that I can literally using wget, mv, scp or other tools to move them to other machine and expect them to just work. To install a binary (not provided by my distro) locally, I only need to copy the result binary into my `~/.local/bin` and it simply works.
deleted 6y ago
[deleted]
javier10e6 6y ago
That is a terrible article. I appreciate statically linked programs that load faster and force the code to behave as a unit vs. a collection of libraries that require the housekeeping of ensuring the correct version the program and the library or else...
taylodl 6y ago
This article is from 2004 - a time when memory and disk were comparatively expensive to the price they are today. Nowadays those cost factors aren't an issue. Package managers are regularly utilized to keep systems up-to-date and developers now regularly build, test, and distribute their software. The changes between now and 2004 are so stark that now we'd consider dynamic linking to be harmful! How times have changed!
- cogburnd02 6y ago
  Hmmmm... https://sta.li/ https://sta.li/
cosmin800 6y ago
I don't know about you, but I always static link when possible because I am so tired of the DLL (SO) hell. Quick real-life example: I had a program that heavily relied upon curl library and fork(), compiled (dynamically linked) and ran in debian 8 would work flawlessly, when ran over debian 9 would have a lot of memory leaks and sometimes SEGVs, when statically linked the program would ran the same way (good) in debian 8 and 9. It took me days to track down and find out that unlike in debian 8 in debian 9 the curl library uses a threaded dns resolver which of course is not a very smart thing to fork() after, so the new curl library had a new feature that messed up the program logic completely. So, no, I would say static linking is way better than dynamic because is CONSISTENT, the library functions you call are guaranteed to behave the way you think they will. Also the ltrace, LD_PRELOAD, etc doesn't legitimize the use of shared libs, why would you debug a release binary?
cosmin800 6y ago
Quick real-life example: I had a program that heavily relied upon curl library and fork(), compiled (dynamically linked) and ran in debian 8 would work flawlessly, when ran over debian 9 would have a lot of memory leaks and sometimes SEGVs, when statically linked the program would ran the same way (good) in debian 8 and 9. It took me days to track down and find out that unlike in debian 8 in debian 9 the curl library uses a threaded dns resolver which of course is not a very smart thing to fork() after, so the new curl library had a new feature that messed up the program logic completely. So, no, I would say static linking is way better than dynamic because is CONSISTENT, the library functions you call are guaranteed to behave the way you think they will. Also the ltrace, LD_PRELOAD, etc doesn't legitimize the use of shared libs, why would you debug a release binary?
08-15 6y ago
Quick point by point: - A bugfix to a library requires every program that uses it to be relinked. It's true for dynamically linked programs, too, they are relinked over and over. The solution to a forgetful admin is a freaking makefile. - Security by obscurity, don't care. But if you care, addresses can be randomized during static linking, which should happen on the target system anyway (see previous point). - Nobody cares anymore. Hard disks are enormous, memory is enormous, a tiny fraction of either is taken up by binaries. Unless you're on a microcontroller, where you don't have enough ram to even consider dynamic linking. - No! Bad Ulrich! Terrible, horrible, awful design! Bad Ulrich! The "features" of glibc that require dynamic linking should be taken out and shot. The stuff that is too complicated to be implemented without dynamically loading code should be moved into a separate process. Just use dbus, it's pretty much needed for everything anyway. Or embed a scripting language. No, not the one implemented by ld.so!! - Related, stop sabotaging statically linked binaries already, will you, Ulrich? - Wrong! You can distribute objects and link on the target system. Which you have to do anyway, see above. - I'm not sure how I feel about those 'hacks'. At any rate, if we want these hacks, we also get the complexity of the dynamic linker, some form of DLL hell and "Warning: gethostbyname() requires the exact version of glibc, etc..." I don't like the tradeoff, maybe because I don'd debug using LD_PRELOAD shenanigans. - Bonus, from the clarification: What, fewer DSOs make startup faster? Didn't you just say, that with prelinking, dynamic linking was as fast as static linking? Did you just make that up? I get the distinct feeling, Ulrich Drepper is on a crusade to eradicate static linking, and these "arguments" taste a lot like motivated reasoning. I don't know what's driving this guy, but things like the terrible implementation of the NSS cause actual problems in practice. Has he just found a shiny toy and refuses to let go of it? Does he get off on knowing that every program in existence runs a script on startup and he's in control of the interpreter (ld.so)? I don't know, but rationally, his arguments make no sense. MUSL is great software, btw.
setheron 6y ago
The whole premise or necessity for Nix & Nixpkgs is that dynamic linking and the FHS is fraught with peril.
alexdowad 6y ago
Good old Ulrich in his best form!

39 points by vortex_ape 6y ago

rvz 6y ago

Given the movement to static linking in compiled languages like LLVM, Go, Rust for deploying self-contained servers and cross-platform single binary executables, I think this guy has clearly lost the argument. Both him and the authors of glibc.

greatgib 6y ago
I don't agree with you. I think that we are on the top of a tide of the usual 'old is new again' hype, but at some point people will realize again all the issues and limitations of that and go back to dynamic linking. In the same way as containers per application is trendy now. So far it we are encountering all the drawbacks and limitations that used to be known long time ago and that once leaded to dynamic loading...
- rvz 6y ago
  > ...but at some point people will realize again all the issues and limitations of that and go back to dynamic linking. Except that for using third-party libraries and distributing apps or games, there is no reason to use dynamically linked libraries here and also in software distribution. I guess if you talked to iOS / Android app developers who complained about dynamically linked third-party libraries that slowed app execution startup times, desktop app developers who wanted a single statically-linked executable that prevents their app being hijacked with a malicious dynamically linked library via LD_PRELOAD or DYLD_INSERT_LIBRARIES or in general developers or users don't want to have to upgrade other dependencies using dynamically linked libraries to install your software, then I guess one can say it would be better to use static binaries here. Who's to say that a hacker can simply modify a .dll, .dylib or .so in your game and enable developer cheats to ruin it. The authors point on LD_PRELOAD alongside with 'security benefits' is nothing more than outdated and has now been intellectually bankrupted.
  - kvez 6y ago
    I am more interested in security for users than for developers. Sure, it's convenient and might be slightly more work for users to mess with your app, but I don't trust every proprietary app developer to be monitoring the CVEs for every library they use. I'm sure there's a ton of multiplayer games out there that statically linked things like libpng and networking libraries, and by now are an open door into the systems of anyone playing online.
    - jcelerier 6y ago
      > I'm sure there's a ton of multiplayer games out there that statically linked things like libpng and networking libraries you know that even when they are dynamically linked, the developer just ships the dlls next to the app right ?
  - saagarjha 6y ago
    > The authors point on LD_PRELOAD alongside with 'security benefits' is nothing more than outdated and has now been intellectually bankrupted. I LD_PRELOAD (and DYLD_INSERT_LIBRARIES) code into every process I spawn from the command line, except setuid executables (and sometimes even then–I jump through the hoops necessary to make this work), because it lets me extend compiled applications in arbitrary ways to better suit whatever I want them to do. Saying that my usecase is "outdated and…intellectually bankrupted" just because you're not aware of the very real and very convenient benefits of function interposing is the real ignorance here.
  - greatgib 6y ago
    Security is a wrong argument. You can have dynamic loading and locked source folders and disabled LD_PRELOAD. At the opposite, remember that each statically loaded exe would have to constantly be updated/recompiled for all the libraries updates that they use or could let a lot of security holes opened for a long time.
    - ghoward 6y ago
      On the flip side, with libraries that use version symbols, you need to update all of the versions, so dynamic linking might have some of the same problem.

w0utert 6y ago

Didn't see one of the IMO biggest downsides of static linking in that list, which is that there is no sane way to prevent symbol clashes if different components of your application need different versions of the same dependency. When using shared linking dynamic libraries can link some of their dependencies statically into the .so, using strictly local symbols, allowing the application to load it even if it already links some other version of the same dependency. We had this problem at my workplace where we developed a component that needed to embed a relatively modern version of the Lua interpreter. When the team that developed the application that used our component tried to link it, they found out that some ancient component they also had been linking in for some completely different and unrelated purpose had a transitive dependency on Lua 5.0, released almost 20 years ago. For backwards compatibility reasons it was not an option to modify the old component to use the newer Lua version, and linking both Lua versions statically into the same binary is impossible. This would never have been a problem if we could have shipped our component as a shared library (which we already build and use for other internal applications), because there the Lua interpreter could have been linked into the shared library privately.

Joker_vD 6y ago
Um, there is this thing called objcopy: --redefine-sym old=new Change the name of a symbol old, to new. This can be useful when one is trying link two things together for which you have no source, and there are name collisions. --redefine-syms=filename Apply --redefine-sym to each symbol pair "old new" listed in the file filename. filename is simply a flat file, with one symbol pair per line. Line comments may be introduced by the hash character. This option may be given more than once. So it's absolutely possible, just ugly.
- w0utert 6y ago
  I’m not sure I understand how this would help? When linking the main application there will be two component .a files linked into it, that both contain references to symbols that should be resolved to something in a liblua.a, so that needs to be linked along as well, but one component needs to resolve them to liblua.a from Lua 5.2, the other to liblua.a from 5.0. Most (but not all) of the Lua symbols from both versions have the same names. How can objcopy resolve this? Multi-pass linking or something like that?

sesuximo 6y ago

What about: - LTO - small libraries/huge pages can link multiple libraries to one page - lower overhead on first call assuming you lazily link

pjmlp 6y ago
Solved by bytecode distribution formats that AOT/JIT compile, and yes there are those for C and C++ as well.
- saagarjha 6y ago
  JIT compilation is not feasible on a number of platforms.
  - pjmlp 6y ago
    True, but AOT compilation at install time like on watchOS or mainframe language environments is.
- sesuximo 6y ago
  Sorry if I’m being daft but can you provide a link? Something that will dynamically do LTO with shared libraries
  - pjmlp 6y ago
    GraalVM/Sulong and CLR devirtualization.

tyho 6y ago

None of this is really relevant when using containers though. The moment you containerise your application, you have effectively statically linked it. Surely a big reason containers became popular is because you can avoid all the massive headaches you get when deploying dynamically linked software.

salawat 6y ago
I don't understand people who like containerization. So now instead of just taking on the onus of maintaining a software library, now I'm going to have to fuck around with maintaining how I expect it to be deployed, and force other people to adhere to that model, thereby forcing my paradigms and alternate system architecture/exotic filesystem and concomitant weirdness onto the user? I don't know, it just never clicked for me. Seems like a solution that appeals to people who like spending most of their time in config/build files than actually using a system, but maybe I'm just a curmudgeon/haven't used it right.
- judge2020 6y ago
  Docker is for applications. Using it in an actual library for a programming language should be limited to setting up the dev environment needed to run tests [for the benefit of consistency]. It makes sense for someone like Discourse to only support running via docker since they want to make sure you're running with the exact same dependencies they're running with which makes issue-reporting and debugging easier.

alanfranz 6y ago

Static Linking vs Dynamic Linking is a choice that has tradeoffs. If you only consider what's good about dynamic linking and ignore its shortcomings, sure: dynamic linking is The Right Choice! But I see no mention of those tradeoffs. Examples: what if the system-provided library is old or has bugs? What if you're delivering to multiple distributions (or even OSes) and you need a consistent behaviour? What if you need different versions of a library?

baby 6y ago
What if you need to instrument these libraries?
cortesoft 6y ago
Yeah, this article seems a bit disingenuous... it says "I will prove there are no benefits to static linking!", but then goes on just to mention positives about dynamic linking. It never even mentions the reasons why people choose static linking, let alone attempt to refute them.
silvestrov 6y ago
also: What if the 1.0.1 version of the library isn't 100% compatible with the 1.0.0 and the app now crashes? Semantic versioning is nice in theory, but you can't rely on everybody getting in 100% correct and not introducing obscure bugs due to slightly different memory layout of structs. It also makes testing apps more difficult as you now doesn't know exactly what code the clients are running as there can be an incredible high number of permutations of library versions. Dynamic linking also makes it impossible to do link-time optimizations (e.g. inlining, dead-strip, locating code near the function that calls it)
rbarrois 6y ago
I'd say that the deal is: - With Dynamic Linking, it's the job of the system admin (typically a distribution maintainer) to ensure that the system-provided library and application are compatible — and add patches if needed; - With Static Linking, the package developer takes control of the whole experience; they deem more important to deliver a working application to the end user than to ensure compatibility with an OS. This makes sense for isolated applications, e.g. video games. For a system's core stack (coretools, X11, etc.), the end user expects to receive security fixes in a timely manner. From the distribution's point of view, it seems simpler to have a single library to patch, recompile and push, than to have the maintainer of each package statically linked against said library release an updated version...

deleted 6y ago

[deleted]

SAI_Peregrinus 6y ago

Dynamic linking provides some benefits: You save disk space and some memory, especially for things that are used by most of the system like libc. You gain the ability to fix some problems in libraries without updating every app that uses the library. Dynamic linking as implemented by most UNIX-Like OSes has a huge drawback: you only have one system copy of any given library, so all your applications are stuck using the same copy. There's no easy way to have multiple versions (say, with different features) of the same library without giving them different names. This isn't really a failing of dynamic linking, it's a failing of the OS library management model. Dynamic linking in general has some other drawbacks: you lose Link-Time Optimization, there's more overhead at load time (and on the first call with lazy linking), and a few other performance-related issues can crop up on occasion. There's also the issue that replacing a dynamically linked library is a lot easier than replacing a statically linked one for an attacker, so dynamic libraries can allow for vulnerabilities to be introduced (though this also is an OS issue, and there are often good mitigations for it).

arminiusreturns 6y ago
I had always hoped the way Gobolinux approached this would start being adopted by other distros.
- SAI_Peregrinus 6y ago
  Yeah, it seems ideal. But I've not used it, yet, mostly because it's such a big change.
- Slackwise 6y ago
  I'm more interested in the Nix/Guix approach, where you get the benefit of exact version linking, but also an "immutable" version controlled OS, so things like Puppet aren't necessary to manage state. (And likewise, deploy small VMs/containers with just the package manager tools without needing Docker as the entire OS state can be rebuilt using a definition file/version hash.)
dfox 6y ago
Almost any ELF based dynamic linker has support not only for versioned library dependencies but also for versioned symbols in these libraries. The issue is mostly that some widely used libraries implement this wrong and to larger extent that people who try to ship linux binaries just link to whatever latest version they have on their build system.

brundolf 6y ago

The "X Considered Harmful" trope has become exhausting ("harmful", even?). Every single technical decision has trade-offs. You might make a case that you (usually) shouldn't do X in cases like Y, or maybe even that you (usually) shouldn't do X in general, but it's a very big statement to say that something is flatly harmful. Very, very few things are so simple. If you aren't Edsger Dijkstra, there's a good chance you're outing yourself as either inexperienced, or willfully ignorant, by claiming them to be so. Yet this meme has encouraged lots of people to do just that, by making dogmatism easy and catchy.

judge2020 6y ago
https://meyerweb.com/eric/comment/chech.html https://meyerweb.com/eric/comment/chech.html https://news.ycombinator.com/item?id=9744916 https://news.ycombinator.com/item?id=9744916

danShumway 6y ago

I wanted to quote a few parts of the article that I considered incorrect, but then realized I would be quoting most of the article. Short version, almost anyone who's ever tried to ship a nontrivial game on Linux should realize this article is wrong. I particularly disagree with the idea that doing any dynamic linking in a program at all means that instantly it's just as non-portable as a fully dynamically linked program. These things are a continuum, there are degrees of fragility -- you can make a program that breaks less often when ported to new systems. See some of Ethan Lee's writing on this topic.[0] On a meta-level, you should be instantly suspicious of any programming advice that includes quotes like: > Conclusion: Never use [thing]! > This has never been the case and never will be the case. In reality, most software techniques have benefits and downsides. If a debate has survived for over a decade, that means that the answer probably isn't so completely universal that you can completely ignore one side. I'm not going to say that there are no black-and-white correct answers in software development, but people who regularly claim to have found them should be regarded with suspicion. On average, they'll often just be oversimplifying things. [0]: https://gist.github.com/flibitijibibo/b67910842ab95bb3decdf89d1502de88 https://gist.github.com/flibitijibibo/b67910842ab95bb3decdf8...

klodolph 6y ago
Games on Linux are a beast, aren’t they! I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application. For example, it is fairly reasonable to require LibSDL2 to be installed. However, if you bundle LibSDL2 with your app, you’re now directly dependent on things like X11 and ALSA.
- AnIdiotOnTheNet 6y ago
  > I think doing this kind of thing successfully requires making good decisions about which dependencies you consider to be part of the system and which dependencies you want to include with your application. Herein lies the problem, because Linux has absolutely no distinction between "system" and "application". Everything is part of the system.
- 6y ago

nkurz 6y ago

I was trying to figure out how old this page was, but I'm still not certain. The headers show a "Last-Modified" of 2017, but it's definitely much earlier than that. I think the main content dates from at least 2004 from when it was on Redhat's site: https://web.archive.org/web/20041221091001/http://people.redhat.com:80/drepper/no_static_linking.html https://web.archive.org/web/20041221091001/http://people.red... The Clarification looks to have been added in late 2006: https://web.archive.org/web/20061119024251/http://people.redhat.com/drepper/no_static_linking.html https://web.archive.org/web/20061119024251/http://people.red...

airocker 6y ago

Dynamic linking has a major versioning problem. We shipped a product that was an .so file. We once spent months chasing a bug in the field where some of the clients would load a previous version of libstdc++ , calls to our .so would crash the application because we were linked against a higher version. Most ABi is backward compatible but not forward compatible. Plus there is the benefit of being more portable.

ed25519FUUU 6y ago

> fixes (either security or only bug) have to be applied to only one place: the new DSO(s). Yes, and all of your bugs can be introduced in a new single place as well! Seriously though. People love static linking because it solves more problems than it introduces, despite its downsides. I will argue that it’s easier to keep a system secure with static linking. You can upgrade the service that is insecure immediately. Or else you need to wait for a Big Bang deployment, and it can be held up by some random thing that isn’t even in the critical path.

maple3142 6y ago

Although I get the point of dynamic linking (security, size...), I still prefer CLI to be written in a language that enable you to easilly build a static binary such as Go, Rust. IMO, the benefit of static binary is that I can literally using wget, mv, scp or other tools to move them to other machine and expect them to just work. To install a binary (not provided by my distro) locally, I only need to copy the result binary into my `~/.local/bin` and it simply works.

deleted 6y ago

[deleted]

javier10e6 6y ago

That is a terrible article. I appreciate statically linked programs that load faster and force the code to behave as a unit vs. a collection of libraries that require the housekeeping of ensuring the correct version the program and the library or else...

taylodl 6y ago

This article is from 2004 - a time when memory and disk were comparatively expensive to the price they are today. Nowadays those cost factors aren't an issue. Package managers are regularly utilized to keep systems up-to-date and developers now regularly build, test, and distribute their software. The changes between now and 2004 are so stark that now we'd consider dynamic linking to be harmful! How times have changed!

cogburnd02 6y ago
Hmmmm... https://sta.li/ https://sta.li/

cosmin800 6y ago

I don't know about you, but I always static link when possible because I am so tired of the DLL (SO) hell. Quick real-life example: I had a program that heavily relied upon curl library and fork(), compiled (dynamically linked) and ran in debian 8 would work flawlessly, when ran over debian 9 would have a lot of memory leaks and sometimes SEGVs, when statically linked the program would ran the same way (good) in debian 8 and 9. It took me days to track down and find out that unlike in debian 8 in debian 9 the curl library uses a threaded dns resolver which of course is not a very smart thing to fork() after, so the new curl library had a new feature that messed up the program logic completely. So, no, I would say static linking is way better than dynamic because is CONSISTENT, the library functions you call are guaranteed to behave the way you think they will. Also the ltrace, LD_PRELOAD, etc doesn't legitimize the use of shared libs, why would you debug a release binary?

cosmin800 6y ago

Quick real-life example: I had a program that heavily relied upon curl library and fork(), compiled (dynamically linked) and ran in debian 8 would work flawlessly, when ran over debian 9 would have a lot of memory leaks and sometimes SEGVs, when statically linked the program would ran the same way (good) in debian 8 and 9. It took me days to track down and find out that unlike in debian 8 in debian 9 the curl library uses a threaded dns resolver which of course is not a very smart thing to fork() after, so the new curl library had a new feature that messed up the program logic completely. So, no, I would say static linking is way better than dynamic because is CONSISTENT, the library functions you call are guaranteed to behave the way you think they will. Also the ltrace, LD_PRELOAD, etc doesn't legitimize the use of shared libs, why would you debug a release binary?

08-15 6y ago

Quick point by point: - A bugfix to a library requires every program that uses it to be relinked. It's true for dynamically linked programs, too, they are relinked over and over. The solution to a forgetful admin is a freaking makefile. - Security by obscurity, don't care. But if you care, addresses can be randomized during static linking, which should happen on the target system anyway (see previous point). - Nobody cares anymore. Hard disks are enormous, memory is enormous, a tiny fraction of either is taken up by binaries. Unless you're on a microcontroller, where you don't have enough ram to even consider dynamic linking. - No! Bad Ulrich! Terrible, horrible, awful design! Bad Ulrich! The "features" of glibc that require dynamic linking should be taken out and shot. The stuff that is too complicated to be implemented without dynamically loading code should be moved into a separate process. Just use dbus, it's pretty much needed for everything anyway. Or embed a scripting language. No, not the one implemented by ld.so!! - Related, stop sabotaging statically linked binaries already, will you, Ulrich? - Wrong! You can distribute objects and link on the target system. Which you have to do anyway, see above. - I'm not sure how I feel about those 'hacks'. At any rate, if we want these hacks, we also get the complexity of the dynamic linker, some form of DLL hell and "Warning: gethostbyname() requires the exact version of glibc, etc..." I don't like the tradeoff, maybe because I don'd debug using LD_PRELOAD shenanigans. - Bonus, from the clarification: What, fewer DSOs make startup faster? Didn't you just say, that with prelinking, dynamic linking was as fast as static linking? Did you just make that up? I get the distinct feeling, Ulrich Drepper is on a crusade to eradicate static linking, and these "arguments" taste a lot like motivated reasoning. I don't know what's driving this guy, but things like the terrible implementation of the NSS cause actual problems in practice. Has he just found a shiny toy and refuses to let go of it? Does he get off on knowing that every program in existence runs a script on startup and he's in control of the interpreter (ld.so)? I don't know, but rationally, his arguments make no sense. MUSL is great software, btw.

setheron 6y ago

The whole premise or necessity for Nix & Nixpkgs is that dynamic linking and the FHS is fraught with peril.

alexdowad 6y ago

Good old Ulrich in his best form!