28 ms·

“Shared libraries are not a good thing in general”

• tediousdemise 6y ago
  I just made a generic version of the title since I see it every so often: Torvalds: [hot take that almost no one else agrees with]
  • pengaru 6y ago
    > Torvalds: [hot take that almost no one else agrees with] Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from. glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Static linking support is explicitly stated as one of musl's core tenets [0]. One can also argue the popularity of containers has in part been due to glibc's dynamic linking requirements, and containers have become the bloated "modern" version of static linked binaries, bundling all the dependencies without eliding all the dynamic linking overhead. Torvalds is definitely not alone in this particular opinion. [0] https://www.musl-libc.org/intro.html https://www.musl-libc.org/intro.html
    • tediousdemise 6y ago
      Maybe I hyperbolized that a bit. I think there are pros and cons to each approach. The uses cases for static vs dynamic are also different. There are compile time disadvantages to static libraries, and there are security disadvantages to shared libraries. But calling one method better than the other is inane.
      • Arnt 6y ago
        Are you aware of the runtime differences? Both time and space. Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. Compiling with -fPIC reserves one register, which isn't a problem unless you're on a register-starved CPU, which is all of them.
        • zozbot234 6y ago
          > Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. You can avoid load-time fixups via prelinking.
        • floatboth 6y ago
          > register-starved CPU, which is all of them Really? None of the 64 bit ISAs seem particularly starved, especially not aarch64.
          • kelnos 6y ago
            I think the parent's point is that all registers are precious, and freeing up even one by avoiding dynamic might be worth doing.
          • Arnt 6y ago
            It's not particularly short of registers, but IIRC even aarch64 is short enough that they implemented register renaming. I agree that taking a register away is a much less pressing issue there than on the x86.
            • CalChris 6y ago
              The purpose of register renaming in the AArch64 microarchitectures is to support out-of-order. By comparison, you don't find register renaming in the in-order A53. https://en.wikichip.org/wiki/arm_holdings/microarchitectures/cortex-a53 https://en.wikichip.org/wiki/arm_holdings/microarchitectures...
              • Arnt 6y ago
                Suppose you had enough registers, ie. so many that there's no real reason to compilers to save. In that case, you could "support" out-of-order execution by adding a couple sentences to the compiler guidance documentation for your CPU, telling them that they'll get better performance on o-o-o cores if they wait as long as practical with reusing registers.
    • AnthonyMouse 6y ago
      > Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from. Vendoring is hugely popular for projects in year zero, because it makes it easier to ship. Until they learn that packaging specific versions of all your dependencies with your project implies taking on responsibility for backporting all security updates to that version of every dependency when upstream is only actively maintaining the latest version. Or you have a project riddled with known published security vulnerabilities, which is what most of those turn into.
      • lanstin 6y ago
        You have two choices here. Keep updating as you go along and amortize the price for so much open source deps inability to avoid breaking change; or don’t do that and then eight years later lead a heroic update the deps project that takes ten people and millions of dollars and some availability hits. You keep the principal of change only one thing at a time the first way and get promotions the second way. Static vs dynamic is just do you want the code owners to be responsible or some centralized ops group to be responsible. Usually ops doesn’t have the insight to do a good job with deps, so it’s better to have dev do it, e.g. statically linked go binaries, statically linked C/c++, frozen package versions in Python, I think mvn has a similar capability, also version pinned base container images etc etc. hence the fact all the packaging systems add new versions and only rarely remove things as that breaks it all worse than a security hole. As a dev person with a lot of ops experience I like static because I don’t want some janky dependency change to come into the execution path when I am not aware of making a change. On the other hand, people usually just freeze everything and then don’t worry about it, which is wrong. But your security team needs to review source and source deps not just deployed versions. So if code is still owned, the bad deps can be found and fixed either way.
        • lanstin 6y ago
          I think a case for dynamic loading of your SSL library can be made.
      • hibbelig 6y ago
    • attractivechaos 6y ago
      > glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Anothre reason is that glibc is equally bad for dynamic linking. Often a binary compiled on recent glibc doesn't work with older glibc. It is certainly possible but quite challenging to compile portable executables on Linux.
  • CrLf 6y ago
    I don't think this is the case here. It's not hard to agree that shared libraries are meant to be _shared_ among different applications and when they're not (or not shared enough, which is a subjective measure) it's best to statically link.
  • incrudible 6y ago
    Imagine a world where Linux binaries "just worked", every distribution's major selling point, the package manager, would be obsolete. These people are propagandizing the bogus security argument that static linking is bad practice. Take the red pill: statically linked executables are actually good practice.
    • kiwijamo 6y ago
      Until the day comes when an application isn't updated with a patched library and people get hacked. This is the reason why I'm not so keen on statically linked applications. I'd rather my applications focus on their concerns and link to shared libraries for stuff like SSL and so on. This means the SSL people can focus on shipping secure SSL libraries and application people can focus on shipping applications built on secure shared libraries.
      • incrudible 6y ago
        This is an unlikely scenario. Most vulnerabilities are not in shared libraries. If you don't update your software, either it doesn't matter, or you eventually run into security issues. Optimizing for the unlikely scenario is not a worthy tradeoff. Focusing on shared libraries can indirectly lead to less security overall, because people run outdated software, because of dependency hell leading people to defer upgrades.
      • CalChris 6y ago
        That application wasn't updated, full stop.
    • Cloudef 6y ago
      While i want static linking, i would still use package manager for convenience and because it provides some form of curation. (Packages in repos are usually not malicious)
      • incrudible 6y ago
        Sure, but again, if these packages were simple binaries that "just work", you wouldn't need your package manager to be strongly coupled to your distribution.
  • coldtea 6y ago
    > Torvalds: [hot take that almost no one else agrees with] [but should and would if they knew better] Here, I fixed it for you!
• phendrenad2 6y ago
  Ah he gets it. Shared libraries are a good idea if you only consider performance and idealized scenarios. In the real world, where users have to use programs that they want to not be broken, vendoring absolutely everything (the way Windows apps do it) is essential.
  • macksd 6y ago
    Huh - are DLL's not a thing anymore? IMO the best reason to use shared libraries has always been cases where updates are likely critical and there's a well-defined interface that is unlikely to have breaking changes. Like libssl. If there's a security fix, I'd love to have `apt update` apply that fix for everything, and not worry about verifying that everything that might have statically linked it gets updated too. Although ideally the latter would be easy anyway. It's also nice for licensing - being able to have dependencies that you don't have to redistribute yourself.
    • city41 6y ago
      Many apps on Windows bring their own local dlls that no other apps on the system use.
    • R0b0t1 6y ago
      To expand on the other answer you got, there's nowhere to put DLLs by default. You install what you need in your directory in Program Files and at that point you may as well statically link them.
      • TeMPOraL 6y ago
        Or you may as well not. DLLs sitting next to your executable are pretty much like static linking, except with one crucial difference - they can still be swapped out by end user if the need arises. For instance, to apply a fix, or to swap out or MITM the DLL for any number of reasons. It's a feature that's very useful to have on those rare occasions when it's needed.
        • 6y ago
• cjohansson 6y ago
  This sounds like an opinion that welcomes Rust and Go since they use static linking or am I wrong?
  • loeg 6y ago
    I think you can dynamic link Rust, but the ABI is unstable. https://doc.rust-lang.org/reference/linkage.html https://doc.rust-lang.org/reference/linkage.html
    • deleted 6y ago
      [deleted]
    • estebank 6y ago
      The Rust ABI is unstable, the C ABI is stable. You can always dynamically link against a C object file, even one written in Rust. I would love to see a restricted stable Rust ABI for types without generics and trait objects and nothing else. That would enable library authors to easily express/provide a dynamic linking surface, while the default remains static linking. I tend to agree that things that "are part of the OS" like QT and GTK should be linked dynamically, while most other things shouldn't.
  • kmeisthax 6y ago
    Kind of. Go already has several other deal-breakers for systems programmers (garbage collected everything, runtime-provided greenthreading, etc). Rust is a lot closer to what Linus would approve for in-kernel development, but it still has a few problems, such as failed allocations panicking the system instead of returning an error that can be handled normally.
    • LukaD 6y ago
      Torvalds and the OP are talking about userland software.
  • _ph_ 6y ago
    Go is by default statically linked. You can however create shared libraries from Go which can be called like any C library and call any shared library.
• Yaa101 6y ago
  Shared libraries are a tradeoff from a time (80's and 90's) where big projects like GUI systems needed to be run on small computers. Today with the cheap memory resources and fast interfacing you are often better off with programs that are statically linked as they have no run dependencies other than the OS.
  • zabzonk 6y ago
    > big projects like GUI systems needed to be run on small computers I'm not sure that I would want all of Windows libraries to be statically linked to all my applications. Sometimes you want static linking - sometimes you don't.
    • ex3ndr 6y ago
      GUI is something like an API, let's take React Native - you just have an api to rendering (via bridge), same for X server, same for almost any GUI framework. This is a very very tiny part of an app. UI libraries with prebuilt controls are already almost always bundled with apps.
    • ratww 6y ago
      Well, in Windows and MacOS you need libraries like kernel32.dll etc to sit in-between userland and kernel because the kernel ABI is not stable, like Linux. Also, Linus talks about this sort of exception in his post: "unless it's some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar)".
      • bjourne 6y ago
        Even desktop libraries are moving towards IPC architectures so not even they require dynamic linking. E.g you don't dynlink a library for the trashcan subsystem, you communicate with some service on the dbus thingy. This makes hot updates possible which you don't get with dynamic linking.
        • ratww 6y ago
• flowerlad 6y ago
  The main supposed benefit is not disk or memory economy, but central patching. The theory is as follows: If a security flaw (or other serious bug) is found, then you can fix it centrally if you use shared libraries, as opposed to finding every application that uses the library and updating each separately. In practice this doesn't work because each application has to be tested against the new version of the library.
  • ttt0 6y ago
    Can't we just link statically but still maintain the list of dependencies as if everything was linked dynamically?
    • johnny22 6y ago
      sure, some packages in some of the popular distros are indeed like that. If the package is important enough (like firefox) and the required dependencies are a bit out of step with what's currently used by the rest of the distribution you will sometimes see that for at least some of the dependencies. Most distros dislike doing so, and scale it back as soon as it becomes technically possible.
      • danShumway 6y ago
        This is odd to me, because surely they have to maintain that list anyway so they know which dependent packages need to be tested before releasing bugfixes? Or is that step just not happening? I just feel like, one of the big points of a package manager is that I can look up "what is program X's dependencies, and what packages rely on it?"
      • ttt0 6y ago
        But they dislike just packages requiring different versions of libraries, right? My point is to do literally everything as it is right now, but simply link it statically. You can still have the same version of a library across all packages, you just build a static library instead of a dynamic one.
    • 6y ago
• userbinator 6y ago
  IMHO it's really the way Linux and other Unix-likes do dynamic linking that's "not a good thing", as well as the "hypermodularisation" that Linus alludes to (the left-pad fiasco would be an extreme example of that.)
  • incrudible 6y ago
    Left-pad wasn't really a fiasco. It briefly broke a couple of people's build. If anything, it showed how resilient the system is. People are actually starting to use NPM as a system package manager. Meanwhile, dependency hell on Linux distributions keeps people stuck on outdated versions of software, because some piece somewhere is either too old or too new.
    • VWWHFSfQ 6y ago
      > It briefly broke a couple of people's build. do you think this is an accurate characterization of what happened
      • incrudible 6y ago
        It's an accurate representation of the actual technical fallout. As for the "this is a sign of the end times" narrative that has been spun around it, it's mostly just for the headlines. I wouldn't even disagree that it is a sign of the end times, but practically speaking, it just wasn't a big deal.
  • marcus_holmes 6y ago
    Windows does DLL's too
    • gsich 6y ago
      True, but I have seen software where the DLL files are in the directory where the EXE lies. So copy+paste from another computer works just fine (in most cases). This does not really work in Linux.
    • 6y ago
• thraway123412 6y ago
  I agree that shared libraries are mostly unneeded complexity, as far as free & open source programs are concerned. I kinda like how they open up opportunities for fixing or tweaking the behavior of broken proprietary binary-only garbage without requiring you to RE and patch a massive binary blob. Of course, in a better world, that garbage wouldn't exist and we'd fix the source instead.
• ttt0 6y ago
  To be fair, LLVM is not just clang. It's also clang++, lld, lldb, clang-format, clang-tidy, clangd and dozens of other utilities that rely on the LLVM library.
• CalChris 6y ago
  I like this (shared libraries are mindlessly overused) but there were two points I didn't understand. And the memory savings are often actually negative Not sure but I think this is for a library with only a single user (ever). The GOT ... and the larger call/ret code cost memory. And the dynamic linking will actually cause a lot _less_ sharing because of all the fixups. I have no idea why fixups cause less sharing.
  • marcus_holmes 6y ago
    Your program uses one function from a library. If that library is statically linked, that one function is compiled into your program. The rest of the library is ignored. If that library is dynamically linked, the entire library is loaded into memory, and the function address is shared with your program. More memory used. The advantage of dynamic linking is if that specific version of the library is already in memory (because another application is using it) then it takes up no additional memory. Less memory used.
    • quotemstr 6y ago
      > If that library is dynamically linked, the entire library is loaded into memory, and the function address is shared with your program. More memory used. False. Shared library pages, like pages in executables, are loaded only on demand. You pay a memory cost for what you use, not what's available.
      • naniwaduni 6y ago
        ... at page granularity.
    • cma 6y ago
      I'm pretty sure Linus was talking about the fixups to the function addresses that happen at load time. Because those pages are modified, they aren't shared with other instances of the same process (unless those processes forked from same parent after the shared library was loaded?), causing more memory usage and less cache locality than static linking: https://en.m.wikipedia.org/wiki/Relocation_(computing) https://en.m.wikipedia.org/wiki/Relocation_(computing)
• encryptluks2 6y ago
  I think he has a good point overall, but at the same time I think there can be an equal benefit of shared libraries when it is likely that the library will be shared among multiple applications. It doesn't have to be a tremendous amount. On Arch, users pick their display manager and windows system, and many Qt applications or GTK applications may share multiple libraries and I can see the benefit of smaller packages and being able to update those libraries separately.
  • kelnos 6y ago
    Torvalds specifically calls out those kinds of shared libraries and considers it reasonable to leave them shared.
    • encryptluks2 6y ago
      He hinted that for base installations it makes sense. GTK or Qt frameworks and libraries aren't part of the base and the are highly dependent on what app you're installing. Usually if you are just a casual Linux user you wouldn't understand this. In these instances they aren't commonly shared across all applications, just maybe 2 or 3.
• deleted 6y ago
  [deleted]
• Jach 6y ago
  Note to 1-bit people: saying that "X is not a good thing in general", is not the same as saying that "X is a bad thing in general". All that's being said here is "tradeoffs exist" and highlighting some issues some people apparently like to ignore. One of my favorite libraries is SDL. Of course, when SDL2 came out, they moved away from the LGPL license to a more liberal zlib license, which allows static linking without infecting the license of your project. Great, except now as years go by and games are abandoned, they stop working in new environments, or at least don't work as well, and you can't just force in a new dynamic version of SDL to save things like you used to be able to. The solution they came up with: even if the program is static linked, the user can define an environmental variable to point to a dynamic lib and that will be used instead. Best of both worlds, at least going forward for programs that use a version of SDL2 that incorporates the feature. (Some more details: https://www.reddit.com/r/linux_gaming/comments/1upn39/sdl2_adds_dynamic_api_magic_to_allow_updating_it/ https://www.reddit.com/r/linux_gaming/comments/1upn39/sdl2_a...)
  • old-gregg 6y ago
    The "1-bit people" part made me laugh. Thanks ;)
    • Karsteski 6y ago
      I'm definitely stealing it
  • AtlasBarfed 6y ago
    I think this is still the hallmark of API volatility. I personally think with the stalling of serial speed in CPUs, we will eventually have to optimize and stabilize interfaces. I think some of that will require stabilization of many APIs and libraries such that they don't change over the span of a decade. This era is basically unthinkable in the current age, but I think that is because we have been living in the "free lunch" era of faster better CPUs every year for too long. Much like all of the 20th century was an era of resource availability. The 21st will be an era of succeeding within resource constraints: with efficiency. Those libraries will be excellent candidates for shared libraries.
• RcouF1uZ4gsC 6y ago
  I think one area where shared libraries is are a huge toll is in C++. ABI concerns have crippled a lot of improvements to the library. In addition, because C++ development by default uses the system C++ standard library, many times the latest C++ languages and libraries can’t be used. In addition, with regards to saved disk space and central patching, Docker negates a lot of those benefits. Basically instead of static linking, you end up shipping a whole container with a bunch of shared libraries
• rvz 6y ago
  Exactly. I don't see the benefits of shared libraries in terms of software distribution. It makes sense for them to be used as standard system libraries, like in macOS with System.dylib or libc.so in many distros. For general apps using third-party libraries, nowadays static libraries is the way to go, as I have said months ago [0]. [0] https://news.ycombinator.com/item?id=24606580 https://news.ycombinator.com/item?id=24606580
• ChiefOBrien 6y ago
  Torvalds: > Disk is cheap I certainly don't want each installed desktop app to have a copy of base gnome/kde runtime and everything down to libc. And the implication is even the graphics would be duplicated, for example the Adwaita icon pack is huge. So if I have a full set of gnome applications (say 50) would I have 50 copies of Adwaita icon set? Suddenly disk space isn't cheap. Shared libs are good and we could do better than flatpaks and containers and static linking. And just because shared libs are PITA it's not just because of their nature, it's the lack of tooling from supposedly modern languages, lack of guidance, lack of care for versioning and API stability, and distro agnostic convention. Each of these problems can be solved by not sweeping them under the rug.
  • somebodythere 6y ago
    I think you can deduplicate at the OS level
    • nix23 6y ago
      Great idea and hold the hashtable of all files in the ram...what a great idea.
      • Sleepytime 6y ago
        Hardlinking them works fine.
        • deleted 6y ago
          [deleted]
        • nix23 6y ago
          That's not de-duplication.
    • rodgerd 6y ago
      ksmd will do that for you today!
  • 6y ago
• superkuh 6y ago
  Shared libraries libraries amplify future shock. Combined with the modern rapid pace of new features being added to libraries and languages themselves even distros that aren't even released yet are already out of date and unable to compile and run $latest application. The symptom of this amplified future shock is the rise of containers on the desktop. The problems of updating applications individually because they're statically compiled are insignificant compared to the problems that come from having to do everything within containers.
  • pengaru 6y ago
    > The problems of updating applications individually because they're statically compiled are insignificant compared to the problems that come from having to do everything within containers. Bundling dependencies aren't the only reason containers are used though. We still want to sandbox everything, and those sandboxes need to provide dependencies. At least with dynamically linked you can decouple and share things like "runtimes" in the flatpak vernacular, which does have some advantages like dependency security updates.
• ufo 6y ago
  As Serge Guelton said in a followup email, one of the reasons that Linux distros use dynamic linking is to reduce the disk usage and bandwidth requirement for updates. Technically speaking, would it be possible to design a Linux distro so that the installed executables are statically linked but an update for a library doesn't require redownloading large executables for every package that depends on the library?
  • wizzwizz4 6y ago
    Yup: distribute source code. (Step 0 is making compilation not require the world's most powerful gaming rig.)
    • mjbrusso 6y ago
      What about distribute LLVM bitcode?
  • drdec 6y ago
    I think a distribution similar to Gentoo, where the software is compiled at install time, could do something like this. (I don't know enough about Gentoo to know if it already does.)
    • brandmeyer 6y ago
      The single number one gripe about Gentoo was how long it took to build packages, closely coupled to the difficulty in keeping a system up-to-date. In response, Gentoo developed methods to ship binaries for the common cases.
  • klyrs 6y ago
    Absolutely possible, with source distribution. It keeps downloads small, and upgrades (notionally) transparent. The downsides are that it's computationally expensive on the user end, in addition to storage bloat. Does the latest firefox depend on the latest llvm? Depending on how awful your hardware is, you might wait a day for those two packages to build.
  • 6y ago
• stephc_int13 6y ago
  In my opinion, this idea of shared libs should have been abandoned at least two decades ago. I see it as a huge weakness on the GNU/Linux world because it's so widespread.
  • ratww 6y ago
    It's also a weakness in other systems IMO. In Windows, "DLL hell" used to be a thing. It's mostly solved by (among other things) Microsoft forbidding installers from overwriting system DLLs. Also today most apps bundle the DLLs along with the apps and don't really share them. In MacOS, brew the popular package manager recently took upon using shared libs for everything. While it mostly works it can cause some version incompatibilities, you tend to accumulate garbage in your disk after uninstalling stuff and now you have to routinely update 3x more packages...
    • stephc_int13 6y ago
      Yeah, the idea is widespread and not unique to the Unix world, but many apps are self-contained on Windows, whereas it is quite rare on Linux, AFAIK. Cascading dependencies are a really bad thing, containers were invented mostly to fix a bad design by adding a new layer... The simple fact that we need an installer is a sign that something is wrong.
• marcus_holmes 6y ago
  this really feels like one of those roundabouts that CS/IT does. "every program the system runs depends on library foo, so why is this loaded into memory x times? Let's load it once and share it" "there are 10 versions of library foo, none of them are backwardly compatible. Every application loads its own version of foo. There are now at least 5 versions of foo in memory at once. Why is the OS trying to manage this? We should statically link every application and stop worrying about it" This has already gone back and forth at least once since I started working in the industry.
  • lambdatronics 6y ago
    I think Unison's core concept[0] is the right way to address this: content-addressable packages/libraries. You can then get the best of both worlds - apps that need the same version of a dependency automatically share it, instead of duplicating it, but there's no chance of mistakenly substituting incompatible versions (barring a hash collision). What about when a security vulnerability in a shared library is found? Each dependent app would need to be explicitly changed to accept the patched version, once compatibility has been established. The problem with trying to force-update all apps by patching the sole version of a shared dependency is that it may break some apps. This runs afoul of the maxim "security at the expense of usability comes at the expense of security." At best, you end up with some hacky dependency-management system, at worst the patch remains all-or-nothing so that the slowest apps veto the update. You can also imagine that the list of versions of a shared library could be maintained in an implicitly declarative fashion, so that old versions with no dependent apps could be garbage-collected using reference-counting. [0]https://www.unisonweb.org/docs/tour/ https://www.unisonweb.org/docs/tour/
• nix23 6y ago
  Time that Linux suckless right?
  • tomxor 6y ago
    Not sure if people are getting the reference here... The "suckless" tools authors are known to prefer static linking, they even tried to make a linux distro "stali" with the philosophy of static linking everything.
    • deleted 6y ago
      [deleted]
• pdkl95 6y ago
  As Linus points out, the utility of shared libraries depends on the situation. Widely used libraries like libxcb, libGL, and zlib that tend to have reasonably stable ABIs are good candidates for sharing, while the "couple of libraries that nobody else used. Literally nobody" should obviously be statically linked. Software engineering is mostly about learning how to choose which method is appropriate for the current project. However, there important advantage of using .so files instead of building into a big statically linked binary is NOT that a library can be shared between different programs! The important benefit is dynamic linking. The ability to change which library is being used without needing to rebuild the main program is really important. Maybe rebuilding the program isn't practical. Maybe rebuilding the program isn't possible because it's closed/proprietary or the source no longer exists. If the program is statically linked, then nothing can happen - either it works or it doesn't. With dynamic linking, changes are possible by changing which library is loaded. I needed to use this ability last week to work around a problem in a closed, proprietary program that is no longer supported. While the workaround was an ugly LD_LIBRARY_PATH hack, at least it got the program to run.
  • brundolf 6y ago
    It seems to me that several circumstances have changed since the idea of dynamic linking first came about: - A whole lot of software is now open and freely available - The code-changes-to-binary-updates pipeline has been greased by a couple orders of magnitude, from internet availability to CI to user expectations around automatic updates (for both commercial and non-commercial software) - Disk space, and arguably even RAM, have become very cheap by comparison Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
    • drewcoo 6y ago
      To underscore the gp's point, I have had to work with a 3rd party-provided .so before. No sources. A binary. And strict (if unenforceable) rules about what we could do with the binary.
      • CyberDildonics 6y ago
• lamontcg 6y ago
  Dynamic linkers aren't really dynamic enough. If I don't use any features that require libbz2 or whatever a piece of software shouldn't complain if libbz2 isn't there on the system. If I add it to the system I should be able to use those features without recompiling and relinking everything. Half of the features of autoconf such as "is this library with this method signature available?" should be moved into the dynamic linker. Software would need to be rewritten to be more lazy and modular, but it'd make dynamic shared libraries actually dynamic. Having to make all those decisions at compile time instead of run time isn't very dynamic.
  • deleted 6y ago
    [deleted]
  • nemetroid 6y ago
    This is perfectly possible with dlopen(), but as you say, requires work on the software side.
    • throwawaaarrgh 6y ago
      And the reason we have "dependency hell" in packaging is because software developers never take advantage of this. If applications would use dlopen() with versioning (a real thing that has existed for a while now) to load the specific version of the library they want, we could install 1,000 different versions of a library in our systems today, no problem at all. For different apps to use different versions of dependencies but still interact with each other, they would need strict rules about how to use interfaces to different versions. You obviously don't want two different programs that use two different versions of a library to talk to one another, because what if version A has a different schema than version B? Unless there was a very specific way to pass along interface information between two programs, so that they could independently handle changes in their interfaces.
  • int3 6y ago
    Don't weak definitions already cover this? It's just a matter of the application authors taking advantage of it...
• smitty1e 6y ago
  It strikes me that packaging applications & dependencies as Docker containers is a huge empirical example of Torvalds' point.
  • jart 6y ago
    Keep in mind Torvalds is replying to a distro person, and distro people like it that way. They're never going to see eye to eye. This piece of software was developed on Fedora 30. Therefore to run the binary you need to install Fedora 30 in a Docker container beforehand. Who doesn't want to ride on coattails? DSOs are like open-lock-in and no native software can escape its grip. These days I'm not even sure if most software can be considered a thing in and of itself that's separable from the specific version of the specific distro on which it's written, due to these libertine development practices using unportable magic .so files as implicit dependencies, since no one wants to write a cmake config vendoring the transitive closure of some C++ library that takes 30 minutes to build.
    • smitty1e 6y ago
      . . .so you build it and the cp it into the Docker image, and cease caring over whether a target platform has a copy because it's all in some container layer, no?
      • jart 6y ago
        That's a big ask for someone who doesn't use Docker. If someone tells me I need it, then I set up a virtual machine. I write binaries that run fine on seven operating systems. Using a single file. If someone comes to me and says their program can only run on a single version of a single distro of Linux then I think that they might benefit from learning my way of doing things.
        • smitty1e 6y ago
          We've moved off of my "empirical example" point by now.
• fxtentacle 6y ago
  From what I observe, most people use dynamic libraries simply because they cannot get stuff setup correctly for linking statically. So they're usually not shared, but merely use the shared library file format. For a practical example, most software on Windows will bring its own folder with DLLs. One step further, this is probably why we're now seeing docker everywhere: it's a Linux equivalent to Windows and Mac app store apps.
  • haolez 6y ago
    There is also licensing. Some licenses won't allow static linking on proprietary software.
• dTal 6y ago
  "Don't abstract your code into functions! Just copy and paste it all over the place. That way, when you need to change the behavior of one of them, the rest don't break!" /s Abstraction is good. Surely anyone can see that having one copy of code is superior - technically, and from a standpoint of elegance - than duplicating it once per program. A huge amount of effort has gone into making this infrastructure work. This isn't just about disk space, or memory, or the ease of patching - it's about composability. I like that I can override behavior with LD_PRELOAD. If we think of our systems as a collection of unrelated, opaque blobs, then static linking looks seductively simple; but if we think of them as a network of interacting code that transcends application boundaries - vivisected and accessible, like a Lisp machine - then static linking is a repulsive act of violence against deeply held aesthetic principles. If dynamic linking doesn't work well - if we've messed it up - that's a sign we should try harder, not give up.
  • jart 6y ago
    You're forgetting the dream of MULTICS. Modern MMUs grant each program dominion over its virtual address space and white picket fence like the American Dream. It doesn't inhibit cooperation between program processes which can totally happen over IPC or network mechanisms. Sort of like how you don't need a farm in your backyard to eat food. You just go to the food store and pay them money. What you appear to want is a collective farm where code is reorganized into a communal state where no boundary is sacred and that MMU you paid good money for isn't worth the silicon on which its printed.
    • CyberDildonics 6y ago
      Reinventing QNX will be cutting edge for decades.
      • jart 6y ago
        Which reinvented 5ESS MERT which was the intended purpose of UNIX, which reinvented MULTICS. It's about more than real time reliable systems though. It's about software freedom too. Ask yourself: "Am I authorized to pick an arbitrary pointer address, and store what I want there?" If the answer is "no, you can't, because some DLL or SO or DYLIB file might have been randomly loaded there" then you are not using a Free as in Freedom operating system.
• eddieh 6y ago
  On desktops this is 100% correct. On mobile it sucks to have to ship stuff statically linked (or even dynamically linked) but included in the .apk or .ipa. But mobile is a different world, it is more like the past where every resource is constrained. You have limited storage (and storage is relatively expensive), limited memory, limited bandwidth (despite what the carriers want the consumers to believe), and the app stores impose limits too.
• quelsolaar 6y ago
  A good reason to use dynamic libraries is for plug-in systems. The application can then dynamically find out what plugins are on the machine and run them. If a plugin isn't installed nothing breaks. It makes it possible to have a open API to a closed application. Say if you have a paint application and you have plugins for file loading, then users can write/install additional file formats as plugins, without the app changing. It can also be helpful in development since you can move dependencies to separate modules. If a file loader requires libpng, then you need that dependency to rebuild the plugin but not the core application. In general I agree with Linus, an application should not be separated in to multiple parts unless it serves the user.
  • kragen 6y ago
    That's what he said in his last paragraph, quelsolaar.
    • quelsolaar 6y ago
      I read it as, modules that are loaded and unloaded dynamically, facilitating things like the ability to update a running application. On second reading you are probably right that plug-in interfaces even if plugins are only loaded at startup, would fall in to the category he describes.
• deleted 6y ago
  [deleted]
• deleted 6y ago
  [deleted]
• ExcavateGrandMa 6y ago
  Everything is not good thing in general...
• DoomHotel 6y ago
  From some of the replies further down that thread, it appears this is not as big a problem as it first seems. First of all, the dynamic linker is slow and not optimized for program startup times. [ https://lore.kernel.org/lkml/4b77489c05f6459ead7233c3fb69f61e@AcuMS.aculab.com/ https://lore.kernel.org/lkml/4b77489c05f6459ead7233c3fb69f61... ] And second, compiling clang with Profile Guided Optimization makes a big difference. [ https://lore.kernel.org/lkml/YItU3YrFi8REwkRA@archlinux-ax161/ https://lore.kernel.org/lkml/YItU3YrFi8REwkRA@archlinux-ax16... ]
  • deleted 6y ago
    [deleted]
• forrestthewoods 6y ago
  I quite like shared libraries. What I don’t like is actually sharing them. I firmly believe that all applications should ship their dependencies. Launching an application should never be more complication than “unzip and double-click executable”. Docker only exists because run-time environments are too complicated, convoluted, and mutually incompatible. It’s insane. > Docker containers are kind of neat. They are also kind of a craven surrender to the rotting mess of excessive software complexity. - John Carmack (https://mobile.twitter.com/id_aa_carmack/status/1385103110977179649 https://mobile.twitter.com/id_aa_carmack/status/138510311097...)
• herf 6y ago
  When you look at the differences in speed between distros, often the main thing is which glibc is being used! Regular string ops can vary in speed by 2-5x based on which old glibc you have.
• quotemstr 6y ago
  Shared libraries are still very important for memory efficiency on mobile systems and in other constrained environments. Android could not work as it does today without the memory savings that sharing libraries gives us. Megabytes matter when you're talking about a device with only a few gigabytes of memory and a hundred processes that all need to run some bit of code or other. It's important to avoid equating "Linux" with "{Desktop,Server} {Ubuntu,Debian,Fedora,SUSE}". There's an entire universe of Linux-using systems out there that have operating constraints very different from your typical anonymous cloud server machine.
  • alerighi 6y ago
    A modern smartphone has more RAM than most computer that people use to do actual work. I mean I see around a lot of people that uses computers with 4Gb of RAM, and the standard nowadays is 8Gb for Android phones. Even with disk space, nowadays a smartphone typically has 128Gb of rather fast SSD, a lot of people still uses computer with spinning disks. Also are shared libraries all that used in Android? The part of the system that is written in native code is very small, it probably doesn't make all that sense to use shared libraries for that.
    • quotemstr 6y ago
      Smartphone memory is used for lots of things, e.g. modem carve-outs, graphics buffers, and page cache. The fraction left over for actual code is pretty small, especially on the smaller devices common in the developing world. Shared libraries are essential for using this memory efficiently.
• nitrogen 6y ago
  Shared libraries and dynamic linking make writing FFI wrappers in other languages possible. If every library were statically linked, I wouldn't be able to write things like this: https://github.com/mike-bourgeous/mb-sound-jackffi https://github.com/mike-bourgeous/mb-sound-jackffi
• ashtonkem 6y ago
  The era where disk space is a rare commodity and the savings gained by dynamic libraries are worth the tradeoffs ended a decade ago. Heck, the heavy usage of Docker or similar in industry really signals that developers are more than happy to tradeoff disk usage, often a lot of it comparatively speaking, for reliability.
• oreally 6y ago
  Does anyone not seem to care about code hotloading at all? Shared libraries enable that. It's a HUGE iteration bonus that seems gets glossed over in favor of onerously processes and dogma.
• a-dub 6y ago
  you watch. everything will go static and then some new runtime hot patch mechanism will be introduced. what does ram usage look like if you build gnome, qt, x, wayland apps statically? seems like you might end up with weird stuff like huge rss for things like calculators and desktop tools. moreover, doesn't it just push compatibility problems to the network protocol/ipc level anyhow? i suppose maybe that's more stable... for now?
• aduitsis 6y ago
  > Pretty much the only case shared libraries really make sense is for truly standardized system libraries that are everywhere, and are part of the base distro. I dare say this also adds credit to the approach of FreeBSD, where the base system itself includes some basic libraries such as openssl, etc. Updating the system patches the libraries uniformly, so any ports using them are automatically patched. And there is no problem with security patches breaking compatibility, because the version of libraries in base remains mostly stable for a particular version of FreeBSD and security patches are backported as necessary. Which is of course what happens in popular long-term support Linux distributions too, albeit at the package level. It is also possible to have ports that use a different version of a library too, though that often requires building custom packages locally or via a centralized Poudriere builder.
• mberning 6y ago
  If I had a dollar for every time I had two JAR files depending on different versions of Jackson...
• whatever1 6y ago
  If an app is actively maintained then the publisher should be able to releash a patch to fix a security hole, regardless of whether that is in its main source code or in the libraries it uses. If it is abandonware, then hoping that shared libraries will save you is at least naive. Currently shared libraries only cause pain to the linux desktop users. I always postpone updating my system because I don't want to deal with the broken dependencies. It is insane how bad the situation is.
  • kiwijamo 6y ago
    I've used Debian as a daily driver on my laptop for several months now and I haven't observed issues with broken dependencies. I've observed that issue with other distros in the past (as well as the *BSDs) due to package managers that don't really have a good approach for dealing with complex changes to dependencies (e.g. the recent change in python default version in FreeBSD could not be handled by the package manager and needed to be done manually following instructions). I believe apt basically solves this problem surprisely well even in the face of complex changes. I think shared libraries are worth the small pain since it means that in the case of updating a library, only the specific library need to be updated with instead of everything that happen to statically link a library needing updating (if the upstream maintainer updates it in a timely manner or even at all!). A world where everything is statically linked would be a world where we were stuck with whatever the upstream maintainer decides is best for them and not what is best for the users.
• gigel82 6y ago
  What about memory mapping, shared memory? Are those not a thing on Linux? I'd say err on the side of dynamic linking whenever there's a chance at least 2 programs that load the library can run at the same time.
• BugsJustFindMe 6y ago
  It strikes me that the Linux position of "we won't provide a stable binary driver interface to force you to mainline your device drivers" is a kind of spiritual opposite to dynamically linking shared libraries. One of the most common arguments in favor of shared libraries is that you can patch libraries without rebuilding everything under the sun, but the mechanism used for that is in fact a stable binary interface that the Linux kernel itself refuses to provide. So, if nothing else, Linus taking this position on shared libraries is extremely consistent.
  • jchw 6y ago
    I do think usermode and kernelmode concerns are different, because the kernel drivers are basically the libraries or extensions, not the program itself. If you statically link a driver with the kernel, you get a kernel. If you statically link Qt with your GUI application, you get your GUI application. The kernel moves fast, because it has to. The usermode does not. The libc interface and relevant syscalls are backwards compatible to a greater degree than would ever be reasonable with kernel drivers. Windows does the best here arguably and drivers still had to break a few times in the past couple decades — not All drivers, but certainly some. I guess the fuller picture to me would need to acknowledge the difference between modular plugins, library dependencies, and programs, as well as the difference between kernel and userland. The latter is definitely important: it’s very rare in userland that your CPU vendor matters over just the ISA itself, but in privileged/supervisor mode, often there are more differences directly exposed, as one example of potential pitfall. I think this sortof exemplifies the increased responsibility of the kernel to keep drivers working through refactors and changing hardware, especially on Linux where the hardware support is extremely diverse. I do still agree with Linus regarding dynamic libraries, but I also think the two cases make sense for different reasons.
    • BugsJustFindMe 6y ago
      > Windows does the best here arguably By a light-year. > and drivers still had to break a few times in the past couple decades Like 2 times in the past, what, 20 years?
• lsllc 6y ago
  I agree! I think the big reason Linux has mostly dodged "DLL hell" is that a large chunk of Linux applications are either built from source on the target system, or installed via packages that were built specifically for the target system. However, if you try to build an application binary, then ship it on more than a handful of Linux distros (and even versions within distros), you'll quickly start running into issues with GLIBC symbols etc. Containers makes this incredibly easy as you're basically shipping exactly what you built your code with. I think this is also a place where languages with package managers & the corresponding ecosystems really shine (e.g. Golang, Rust etc). These do well because they are building from source against runtimes that don't involve a pile of shared libraries as you would be doing in C/C++.
  • bscphil 6y ago
    > I think the big reason Linux has mostly dodged "DLL hell" is that a large chunk of Linux applications are either built from source on the target system, or installed via packages that were built specifically for the target system. Correspondingly, that's a non-technical thing to appreciate about shared libraries: they ended up creating the environment where building programs from source or installing them from maintainer provided packages was the only way to do things. If we end up moving to languages where static linking is the norm (or the only option), ecosystem shift away from the maintainer + source based approach might be unavoidable. To my mind, that's pretty clearly a bad thing. It's how you end up with the node ecosystem's security problems, not something as reliable as most Linux desktops have been for decades.
• malkia 6y ago
  There is a whole lot of industries based around shipping plugins. While some of these might work as external processes, they possibly still require some bridge to be loaded as .dll, .so, .dylib. Also security updates. Other than that, static libs for the win. Other examples, speeding up "link" times - Chrome uses component builds - e.g. bundle a bunch of C++ code as .dll and compile/load only that, rather than relink the whole .exe - slower to start and use, but apparently allows you faster development. For what we do, we are using .dlls to enforce that certain C++ code (libs) are not suddenly using symbol from another lib if this was not intended (granted this could've been done if we compiled single .exe for the said lib, but gets the point). We also have mode where we use these .dlls instead of them being statically linked (having both).
  • floatboth 6y ago
    > apparently allows you faster development If linking has such a huge impact on Chromium dev cycles, maybe they should make their codebase less of a bloated hog :) Linking all of Firefox into one libxul.so with LLD takes basically no time (on a good machine) for a release build. Slightly noticeable with all the debug info but still pretty quick.
• dathinab 6y ago
  I can fully agree with Linus there (in general bad, exception are very widely used libraries, and maybe .so's as dynamic extension system). Even if we look at security it in my experience doesn't help much in practice (besides the same exceptions of widely used system libraries). Instead it can even reduce security by introducing subtitle bugs due to unexpected interactions between the dependency and the dependent because the versions of both never had been meant to be used with each other. And at least wrt. open source it's always possible to just recompile the software with patches. Which brings me to another kind of software where shared libraries make sense: Proprietary likely to not get proper maintained software with non-proprietary dependencies.
• deleted 6y ago
  [deleted]
• nh2 6y ago
  For anyone considering contributing to tools and distros that support static linking, check out my StackOverflow answer "Static linking is back on the rise" [1]. [1] https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead/56771892#56771892 https://stackoverflow.com/questions/3430400/linux-static-lin...
• pabs3 6y ago
  Are there any Linux distros that do not support dynamic linking? It would be interesting to see how well a statically linked distro works.
  • muth02446 6y ago
    There used to be https://sta.li/ https://sta.li/ but it appears to be down now
• pabs3 6y ago
  Interestingly, there are folks working on turning Linux into a unikernel, which means you would statically link each application and its libraries with the Linux kernel and then boot that application as a VM. https://github.com/lkl https://github.com/lkl
• fctorial 6y ago
  "Linux is OK, GNU/Linux is a dumpster fire" His way of saying "what you're referring to as Linux is actually GNU/Linux".
• mike_ivanov 6y ago
  I don't disagree with the Linus' sentiment. What I find interesting though, is that some people here are like "we don't need no shared libs, away with them!" Let's suppose this happens. I'm wondering how all those Python, Ruby, Lua and whatnot programs would use database drivers, xml parsers, protocol interfaces, GUI frameworks, etc - all that stuff they are currently loading as shared libs? What are the alternatives?
  • cozzyd 6y ago
    Yeah, not to mention interpreted/jitted C++ is a thing (with cling/ROOT). It's very useful to be able to load random shared libraries and do things with them.
  • Accujack 6y ago
    The same code can be statically linked into the binaries for those programs, or the code can be loaded as shared objects (modules, essentially).
    • mike_ivanov 6y ago
      1. Does it mean that I need to recompile the whole Python or Ruby interpreter each time I want to use MySQL? 2. How am I supposed to load a shared object if there are no shared libraries anymore? This is literally the question I've asked: is there any other mechanism for achieving 2. other than shared libs?
      • SAI_Peregrinus 6y ago
        Shared libraries aren't the same thing as dynamic linking. They're a specific subset of dynamically linked libraries. You can have a single-use .so, and just not install it in the system libs directory. Windows eventually got this right. There are a set of OS-provided DLLs, that are true shared libraries. Then applications wanting to use other libraries bundle their own DLLs. Applications don't generally need to screw with the system libraries, and shouldn't be allowed to (exceptions for some drivers and similar).
• synergy20 6y ago
  Linus works for LF where most of its key members are server and PC and high-end cpu vendors, where resource is plenty so you can run, say 100 static programs. There is a different market called embedded devices, 64MB-256MB RAM are still very common and there are probably 100x more devices in this segment than LF members' fat ones combined. Without shared library, you will quickly run out of RAM/Storage space on them.
  • thr0w__4w4y 6y ago
    Yes and in fact, the majority of embedded systems are actually running not with 64-256MB of RAM, but 64K-256K of RAM. For example, the STM32 line of Cortex M microcontrollers (MCUs) usually have between 32-256K of RAM, and 128K to 2MB of on-chip flash. Different world from a multi-core 1.5 GHz infotainment system (which is arguably embedded), which in turn is different from a laptop with 32GB of DDR4 or a server with .... well, you know....
    • synergy20 6y ago
      yes those are not for Linux anymore and they're even more than Linux devices.
  • shandor 6y ago
    Embedded systems are usually purpose-built instead of the general desktop usage where users install whatever they want, leading to a much smaller amount of processes running simultaneously and effectively removing the benefits of dynamic linking. Just like Linus for PC, I think it sensible to consider libc and other to be dynamically linked also on embedded systems, but maybe even less for other software.
• cryptonector 6y ago
  This again. Yes, static linking is better in every way except semantics, because the bloody linker is stcuk with 1982 semantics while ELF has gotten way more interesting.
• throwawaaarrgh 6y ago
  Why do people still listen to this moron? He doesn't seem to know people use computers outside of kernel development. > the disk savings are often not all that big If you're Linus Torvalds, and we're not talking about Go apps, and other fucking ridiculous 100MB+ static binaries, or any application continuously and iteratively developed and deployed in a CI/CD system, where disk space rapidly grows, and you want to cache as much as possible, and the larger the files are, the slower development gets. > and disk is cheap If you're Linus Torvalds, and you don't need to actually scale (or pay for) systems' disk size and performance (or network egress) in response of being thrashed by random access of gigantic files. And you're not developing for embedded systems. > And the memory savings are often actually negative If you're Linus Torvalds, and you don't have a bunch of giant apps that you need to run in parallel that happen to use a lot of memory. > Pretty much the only case shared libraries really make sense is for truly standardized system libraries that are everywhere, and are part of the base distro. Or for any application that uses any kind of library where you might want to release a security patch quickly. Which is literally every library. If OpenSSL has a hole in it, the fastest way to patch every system in the world is to patch the OpenSSL library. If every OpenSSL app in the world were statically compiled, then every single OpenSSL-using app in the world would need to be patched, tested, released, and upgraded, independently. That's something like 100 different apps for the average Linux host (or probably 10,000+ for an entire Debian tree). Or if you want to do the same for bugfixes. Or you want to release new functionality. Or you want to replace one library with another using the same ABI. We can complain all day about packaging decisions, but shared libraries have a dozen great uses, and static compiles have only two great uses: less dependencies and, and..... and.................. I dunno, we'll make one up. Less packaging work. Which is just from less dependencies. Shit. Oh well, I'm sure we'll come up with a second reason static libraries are good.
  • CrendKing 6y ago
    Doesn't OpenSSL fall into the "truly standardized system libraries that are everywhere, and are part of the base distro" category? You just proved his idea right yourself ;-)
  • matthewts 6y ago
    > If OpenSSL has a hole in it, the fastest way to patch every system in the world is to patch the OpenSSL library. If every OpenSSL app in the world were statically compiled, then every single OpenSSL-using app in the world would need to be patched, tested, released, and upgraded, independently. Even if we upgrade a shared library and the reverse dependencies do not require a rebuild, we should still re-test everything (I think Debian does this already).
• no_wizard 6y ago
  Maybe it’s because I don’t do any C/C++ programming but when you’re building your application can’t you just link your libraries via an arbitrary folder? Why are you dependent on system versions at all? To me that seems horrid. Unless I’m coding directly on the system (i.e. working on the OS itself) can’t you just skirt issues with different versions by just downloading your libraries to an arbitrary folder and pointing your tool chain there?
  • Gibbon1 6y ago
    Far as I know you can do that, applications will do things like that for non-system libraries. System libs can be problematic because the OS's API may change between versions or different flavors of the OS.
• aparsons 6y ago
  Linus is a brilliant guy. Underneath his quirky, cold persona, and the short temper, is a guy I admire not just for his technical brilliance, but his ability to put it into words so concisely and correctly. That’s the one superpower people don’t realize makes great engineers (now remember that English isn’t Linus’ first language). I’ve felt this with Linus, Dave Cutler and other “10x” engineers. Their code really isn’t that much better than anyone else with similar level of experience. Lot of engineers come up with neat solutions. But listening to these men or reading something they’ve written on their topic of interest, it feels as if you’d figured out the thing they’d just explained by yourself.
  • CRConrad 6y ago
    > (now remember that English isn’t Linus’ first language). Nor even his second; I'm fairly sure it's his third (at least chronologically). Why? As a Finland-Swede -- one of the Swedish-speaking minority in Finland -- he would have learnt Swedish as his mother tongue at home (and probably kindergarten / pre-school, if he went to any), and Finnish as his second, at the latest in elementary school. Perhaps in kindergarten / pre-school -- bilingual ones were rarer in his day, but some people put their kids in that of the "opposite" language explicitly to give them a head start on that. Then in around second to fourth grade he would have taken his first foreign language, which for an overwhelming majority is English (though it could also be German or French or something, and nowadays even more alternatives, like Spanish and Russian and whatnot, are available at least at central schools in larger cities). After that he may well have taken another foreign language in parallel, I have no idea (in primary school, up to and including grade 9, that is). I think in his school days, the 1970s / early 80s, that would statistically most likely have been German. English being his third in stead of second language is not necessarily a disadvantage; on the contrary, I suspect it's actually an advantage for him to be at least tri- in stead of just bilingual: Knowing more languages in and of itself may make you more eloquent in all of them. Not knowing him personally[1], so wild-ass guess: I'd tend to think he regards English as his second-but-close-to-first language now, having lived in the USA for so long. Probably still thinks -- to the extent one does that in any language at all -- and dreams in Swedish, and speaks it with his wife, but perhaps at least as much English with his kids who are born there. At least that's how it works for me. Disclosure: second-hand knowledge of the Finnish school system by having a kid in it, first-hand experience of immigration twice over. ___ [1]: Met him once, about twenty years ago, when he signed a copy of Just for Fun for me.
• KingMachiavelli 6y ago
  While I'm not sure I agree 100% with Fedora's policy that everything(?) must by dynamically linked... I think it has a lot of benefits besides the actual reasons for dynamic linking. I'm not an expert but I have observed a few things over the years using and building packages. First off, dynamic linking does not seem to make updating software impossible. There are several rolling release Linux distros (Archlinux, Debian-unstable, Void, Gentoo, etc.) and basically everything works while releasing upstream updates within 2-4 weeks typically. Most are maintained by volunteers. The number of core packages that break due to shared libraries is single digits in my experience of Archlinux over the last 5-8 years. Sure some things are shipped as pre-compiled packages which just come with their own library directories so they can load the versions they need. I will say that I have encountered software that was library hell. This software tended to be academic in nature and use really poor build processes like having dozens of make files while being developed on ancient versions of Ubuntu. The software was not vaporware or abandoned it was just poorly developed. Even if static linking would help - the developers choose not to use it out of ignorance. Other software that has issues with libraries tend to be very large applications where the overhead of shipping an alternative LD_LIBRARY_PATH or 'Docker' container is minimal and has other benefits as well. I cannot imagine how many projects will actually keep their libraries up to date. Electron apps are already often using old Electron versions causing issues where the Electron version won't support Wayland as well as the browser version. I have stopped using the desktop versions of apps like Microsoft Teams and Slack because of this issue. If large companies can't be bothered to update their flagship desktop applications libraries - how can we expect small, open source projects to do better? They no doubt use community feedback from distro maintainers & users to maintain compatibility with upstream libraries. Plus if static linking is used to an extreme amount then their will be N-~1 more versions to 'maintain'. Any popular library/dependency will become a defacto supported version; just look how long it took apps to move to Python 3. Heck, the 'chromium' package seems to still require Python 2 as a build dependency. Even 'Cellebrite', as evil as the program is, was hacked in part because it shipped with out of date libraries and it is probably has a multi-million dollar price tag. TLDR: Developers of all levels suck at keeping dependencies updated. The distro system & build process provide s a much needed barrier between application development and sanity. It's basically the final code review for the Linux & open source ecosystem. That said static linking certainly has its place but also has its own set of issues.
• slver 6y ago
  Shared libraries would be good if there were better semantics in how they're used. Saying "shared libraries bad" instead of working on fixing those semantics is unfortunate.
• rwmj 6y ago
  > We hit this in the subsurface project too. We had a couple of libraries that nobody else used. Literally nobody. But the Fedora policy meant that a Fedora package had to go the extra mile to make those other libraries be shared libraries, I'm a Fedora packager and I'm not sure what he's talking about here. If a single package uses some code then there's no requirement to make that code into a shared library.
  • exciteabletom 6y ago
    The Fedora docs[0] say: > Packages including libraries SHOULD exclude static libs as far as possible > In general, packagers SHOULD NOT ship static libraries. [0] https://docs.fedoraproject.org/en-US/packaging-guidelines/#packaging-static-libraries https://docs.fedoraproject.org/en-US/packaging-guidelines/#p...
    • rwmj 6y ago
      But that's if the libraries are going to be used by another package (and even then it's not a prohibition - packagers should use common sense). Linus said this applied to a single package so it wouldn't need to ship *.a files at all, as everything would presumably be linked to binaries. I'm going to check out this subsurface diving program to find out what really happened here.
• phendrenad2 6y ago
  Shared libraries are a necessity on Linux, because Linux users pride themselves on the efficiency of their systems, and if two programs used separate copies of the same library, that would be wasteful. The exception that everyone could get onboard with would be a common API that all programs use. So, for instance, if everyone could agree to use GTK instead of Qt, then GTK could just be shipped as standard on all Linux distros and there would be no problem of bloat - all programs need GTK. But that goes against the Linux philosophy of modularity & configurability, so that will never happen either. So you need to strike a happy medium. Make an attempt to use a shared library, but if your users don't have it, download it live.
• specialist 6y ago
  Linus writes: "Yes, it can save on disk use, but unless it's some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar), the disk savings are often not all that big - and disk is cheap." How much savings? Has this ever been measured? Just curious.
• greatgib 6y ago
  Forgetting the past, and doing the same mistakes... Not Linus, but a lot of young dev and newcomers are discovering static libs building with GO and co, and think that it is a revolution and that it solves all their problems. But, they are probably forgetting or not knowing the world of static libs before dynamic one became a thing. A big part of the revolution of linux success in servers and all come from the magic that brought a full system built on shared dynamic libraries. Do anyone remember having to install a 1Gb software to setup a hp printer on windows? The full afternoon or day needed to wait for the installation of visual studio? The ms office installation needing multiple gigabytes when openoffice could be downloaded and installed for a hundreds megabytes at most?
  • deleted 6y ago
    [deleted]
• EuAndreh 6y ago
  Are there downsides when library writers build and publish both .so and .a files, and the library users can pick if they want to link statically or dynamically? If so, which downsides are those? I can't tell if they exist, and it seems to me that this is a flexible solution that gives the power of choice to the consumer of the library.
• xondono 6y ago
  I’ve said this before, and even looked into it, but my abilities with web/app development are non existent. Can we get a app/service to read these in some sensible way? That would be awesome.
• deleted 6y ago
  [deleted]
• liamwberley 6y ago
  Liamwberley2012@gmail.com
• bfrog 6y ago
  The result of using shared libraries for absolutely everything is docker/snappy, which are both useful on a utilitarian level but also terrible. Every docker has its own openssl version. Which means every openssl cve means updating every docker image. Awful. It used to be you'd target RHEL X and be done for 10 years.