4 ms·

Oh, yes, I certainly would. Shared libraries are an expensive, trouble-prone solution for that relatively uncommon problem. Anyway that's not why we have share

Oh, yes, I certainly would. Shared libraries are an expensive, trouble-prone solution for that relatively uncommon problem. Anyway that's not why we have shared libraries. They were invented so that memory managers could load the same library once, then map it into the address space for every process which used it. This kind of savings mattered back when RAM was expensive and executable code represented a meaningful share of the total memory a program might use. Of course that was a very long time ago now.

benreesman 4y ago
Also: text pages are shared in the typical case. It was largely a spurious argument to begin with. If it didn't fuck over `musl` et. al. and `libstdc++` it would have been relegated to corner cases years ago.
- comex 4y ago
  Is your first sentence missing a negation, or can you elaborate?
  - benreesman 4y ago
    I should have been more precise, but that can become a treatise by HN comment standards easily so consider there to be an asterisk next to all of this for myriad platform/toolchain corner cases. To the extent that shared libraries economize on memory, or are claimed to, it's when a lot of different processes link the same version of the same library, e.g. your `libc` (or the original motivating example of X). The tradeoff here is that the kernel has an atomic unit of bringing data into RAM (AKA "core" for the old-timers), which is usually a 4kb page, and shared libraries are indivisible: you use `memcpy` and you get it and whatever else is in the 4kb chunk somebody else's linker put it in. Not a bad deal for `memcpy` because it's in everything, but a very bad deal for most of the symbols in e.g. `glibc`. The counter-point is that with static linking, the linker brings in only the symbols you use. You do repeat those symbols for each `ELF`, but not for each process. This is often (usually?) a better deal. A little poking around on my system to kind of illustrate this a bit more viscerally: https://pastebin.com/0zpVqA0r https://pastebin.com/0zpVqA0r (apologies for the clumsy editing, it would be too long if I left in every symbol from e.g. `glibc`). Admittedly, that's a headless machine, and some big clunky Gnome/Xorg thing might tell a different story, but it would have to be 100-1000x worse to be relevant in 2022, and almost certainly wouldn't offset the (heinous) costs. And while this is speculation, I think it would be negative worse, because there just aren't that many distinct `ELF`'s loaded on any typical system even with a desktop environment. Chromium or whatever is going to be like, all your RAM even on KDE. There are couple of great CppCon talks that really go deep here (including one by Matt Godbolt!): https://www.youtube.com/watch?v=dOfucXtyEsU https://www.youtube.com/watch?v=dOfucXtyEsU, https://www.youtube.com/watch?v=xVT1y0xWgww https://www.youtube.com/watch?v=xVT1y0xWgww. Don't be put off by the fact it's CppCon, all your GNU stuff is the same story.
arinlen 4y ago
> Oh, yes, I certainly would. Shared libraries are an expensive, trouble-prone solution for that relatively uncommon problem. This personal assertion has no bearing in the real world. Please do shine some light on what exactly do you interpret as "expensive", and in the process explain why do you feel that basic software design principles like modularity and the ability to fix security vulnerabilities in the whole system by updating a single shared library is "trouble-prone". I mean, the whole world never saw a real world problem, not the UNIX comunity nor Microsoft and it's Windows design nor Apple and it's macOS design not anyone ever, but somehow here you are claiming the opposite.
- jen20 4y ago
  > the whole world never saw a real world problem, not the UNIX comunity nor Microsoft and it's Windows design "DLL hell" [1] was absolutely a thing. It was solved only by everyone shipping the world in their application directories. [1]: https://en.wikipedia.org/wiki/DLL_Hell https://en.wikipedia.org/wiki/DLL_Hell
  - naasking 4y ago
    Not to mention dynamic linking is brutally slow. I don't think people even realize how many cycles are wasted. The notion that dynamic linking never had any real problems and still doesn't is totally false.
- jiggawatts 4y ago
  > basic software design principles like modularity A single product can have a modular design, but modularity across unrelated products designed by different designers is a different concept that should have a different name. In my opinion it’s just not something that really works. This is similar to the “ability” of some software to have multiple pluggable database engines. This always yields a product that can use only the lowest common denominator… badly. But now imagine having to coordinate the sharing of database engines across multiple relying products! So upgrading Oracle would break A, B and fix C, but switching A and B to MySQL would require and update to MySQL that would break D, E and F. That kind of thing happens all the time and people run away screaming rather than deal with it. Shared libraries are the same problem.
Groxx 4y ago
Uncommon by all uses of shared libraries maybe, but certainly not uncommon for security-oriented libraries. A couple times per year is WAY more than often enough to be worth shared libraries, as otherwise you're talking about rebuilding much of the world multiple times per year per security-oriented library. And nearly all libraries eventually become "security-oriented libraries", given enough time. Everything is a hacking target eventually.
SuperSandro2000 4y ago
In real world this would just mean that more bugs are for longer unpatched.

benreesman 4y ago

Also: text pages are shared in the typical case. It was largely a spurious argument to begin with. If it didn't fuck over `musl` et. al. and `libstdc++` it would have been relegated to corner cases years ago.

comex 4y ago
Is your first sentence missing a negation, or can you elaborate?
- benreesman 4y ago
  I should have been more precise, but that can become a treatise by HN comment standards easily so consider there to be an asterisk next to all of this for myriad platform/toolchain corner cases. To the extent that shared libraries economize on memory, or are claimed to, it's when a lot of different processes link the same version of the same library, e.g. your `libc` (or the original motivating example of X). The tradeoff here is that the kernel has an atomic unit of bringing data into RAM (AKA "core" for the old-timers), which is usually a 4kb page, and shared libraries are indivisible: you use `memcpy` and you get it and whatever else is in the 4kb chunk somebody else's linker put it in. Not a bad deal for `memcpy` because it's in everything, but a very bad deal for most of the symbols in e.g. `glibc`. The counter-point is that with static linking, the linker brings in only the symbols you use. You do repeat those symbols for each `ELF`, but not for each process. This is often (usually?) a better deal. A little poking around on my system to kind of illustrate this a bit more viscerally: https://pastebin.com/0zpVqA0r https://pastebin.com/0zpVqA0r (apologies for the clumsy editing, it would be too long if I left in every symbol from e.g. `glibc`). Admittedly, that's a headless machine, and some big clunky Gnome/Xorg thing might tell a different story, but it would have to be 100-1000x worse to be relevant in 2022, and almost certainly wouldn't offset the (heinous) costs. And while this is speculation, I think it would be negative worse, because there just aren't that many distinct `ELF`'s loaded on any typical system even with a desktop environment. Chromium or whatever is going to be like, all your RAM even on KDE. There are couple of great CppCon talks that really go deep here (including one by Matt Godbolt!): https://www.youtube.com/watch?v=dOfucXtyEsU https://www.youtube.com/watch?v=dOfucXtyEsU, https://www.youtube.com/watch?v=xVT1y0xWgww https://www.youtube.com/watch?v=xVT1y0xWgww. Don't be put off by the fact it's CppCon, all your GNU stuff is the same story.

arinlen 4y ago

> Oh, yes, I certainly would. Shared libraries are an expensive, trouble-prone solution for that relatively uncommon problem. This personal assertion has no bearing in the real world. Please do shine some light on what exactly do you interpret as "expensive", and in the process explain why do you feel that basic software design principles like modularity and the ability to fix security vulnerabilities in the whole system by updating a single shared library is "trouble-prone". I mean, the whole world never saw a real world problem, not the UNIX comunity nor Microsoft and it's Windows design nor Apple and it's macOS design not anyone ever, but somehow here you are claiming the opposite.

jen20 4y ago
> the whole world never saw a real world problem, not the UNIX comunity nor Microsoft and it's Windows design "DLL hell" [1] was absolutely a thing. It was solved only by everyone shipping the world in their application directories. [1]: https://en.wikipedia.org/wiki/DLL_Hell https://en.wikipedia.org/wiki/DLL_Hell
- naasking 4y ago
  Not to mention dynamic linking is brutally slow. I don't think people even realize how many cycles are wasted. The notion that dynamic linking never had any real problems and still doesn't is totally false.
jiggawatts 4y ago
> basic software design principles like modularity A single product can have a modular design, but modularity across unrelated products designed by different designers is a different concept that should have a different name. In my opinion it’s just not something that really works. This is similar to the “ability” of some software to have multiple pluggable database engines. This always yields a product that can use only the lowest common denominator… badly. But now imagine having to coordinate the sharing of database engines across multiple relying products! So upgrading Oracle would break A, B and fix C, but switching A and B to MySQL would require and update to MySQL that would break D, E and F. That kind of thing happens all the time and people run away screaming rather than deal with it. Shared libraries are the same problem.

Groxx 4y ago

Uncommon by all uses of shared libraries maybe, but certainly not uncommon for security-oriented libraries. A couple times per year is WAY more than often enough to be worth shared libraries, as otherwise you're talking about rebuilding much of the world multiple times per year per security-oriented library. And nearly all libraries eventually become "security-oriented libraries", given enough time. Everything is a hacking target eventually.

SuperSandro2000 4y ago

In real world this would just mean that more bugs are for longer unpatched.