6 ms·

> Torvalds: [hot take that almost no one else agrees with] Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you'r

> Torvalds: [hot take that almost no one else agrees with] Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from. glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Static linking support is explicitly stated as one of musl's core tenets [0]. One can also argue the popularity of containers has in part been due to glibc's dynamic linking requirements, and containers have become the bloated "modern" version of static linked binaries, bundling all the dependencies without eliding all the dynamic linking overhead. Torvalds is definitely not alone in this particular opinion. [0] https://www.musl-libc.org/intro.html https://www.musl-libc.org/intro.html

tediousdemise 6y ago
Maybe I hyperbolized that a bit. I think there are pros and cons to each approach. The uses cases for static vs dynamic are also different. There are compile time disadvantages to static libraries, and there are security disadvantages to shared libraries. But calling one method better than the other is inane.
- Arnt 6y ago
  Are you aware of the runtime differences? Both time and space. Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. Compiling with -fPIC reserves one register, which isn't a problem unless you're on a register-starved CPU, which is all of them.
  - zozbot234 6y ago
    > Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. You can avoid load-time fixups via prelinking.
  - floatboth 6y ago
    > register-starved CPU, which is all of them Really? None of the 64 bit ISAs seem particularly starved, especially not aarch64.
    - kelnos 6y ago
      I think the parent's point is that all registers are precious, and freeing up even one by avoiding dynamic might be worth doing.
    - Arnt 6y ago
      It's not particularly short of registers, but IIRC even aarch64 is short enough that they implemented register renaming. I agree that taking a register away is a much less pressing issue there than on the x86.
      - CalChris 6y ago
        The purpose of register renaming in the AArch64 microarchitectures is to support out-of-order. By comparison, you don't find register renaming in the in-order A53. https://en.wikichip.org/wiki/arm_holdings/microarchitectures/cortex-a53 https://en.wikichip.org/wiki/arm_holdings/microarchitectures...
        Arnt 6y ago
        Suppose you had enough registers, ie. so many that there's no real reason to compilers to save. In that case, you could "support" out-of-order execution by adding a couple sentences to the compiler guidance documentation for your CPU, telling them that they'll get better performance on o-o-o cores if they wait as long as practical with reusing registers.
        CalChris 6y ago
        They've tried variations of this. It takes instruction bits to describe these registers: 32 regs is 5 bits, target + src1 + src2 is 15 bits already. They've tried scratch pads. The Mill CPU's Belt is yet another approach. https://en.wikipedia.org/wiki/Scratchpad_memory https://millcomputing.com/ How to Use 1000 Registers https://caltechconf.library.caltech.edu/200/ 1000 registers would be 10 bits. So they tried register windows. These are all good ideas but they get to compete. So far, I'm a huge AArch64 fan but I now see the purpose of RISC-V. I watched Chris Lattner's ASPLOS talk and it finally clicked. https://www.youtube.com/watch?v=4HgShra-KnY
AnthonyMouse 6y ago
> Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from. Vendoring is hugely popular for projects in year zero, because it makes it easier to ship. Until they learn that packaging specific versions of all your dependencies with your project implies taking on responsibility for backporting all security updates to that version of every dependency when upstream is only actively maintaining the latest version. Or you have a project riddled with known published security vulnerabilities, which is what most of those turn into.
- lanstin 6y ago
  You have two choices here. Keep updating as you go along and amortize the price for so much open source deps inability to avoid breaking change; or don’t do that and then eight years later lead a heroic update the deps project that takes ten people and millions of dollars and some availability hits. You keep the principal of change only one thing at a time the first way and get promotions the second way. Static vs dynamic is just do you want the code owners to be responsible or some centralized ops group to be responsible. Usually ops doesn’t have the insight to do a good job with deps, so it’s better to have dev do it, e.g. statically linked go binaries, statically linked C/c++, frozen package versions in Python, I think mvn has a similar capability, also version pinned base container images etc etc. hence the fact all the packaging systems add new versions and only rarely remove things as that breaks it all worse than a security hole. As a dev person with a lot of ops experience I like static because I don’t want some janky dependency change to come into the execution path when I am not aware of making a change. On the other hand, people usually just freeze everything and then don’t worry about it, which is wrong. But your security team needs to review source and source deps not just deployed versions. So if code is still owned, the bad deps can be found and fixed either way.
  - lanstin 6y ago
    I think a case for dynamic loading of your SSL library can be made.
- hibbelig 6y ago
attractivechaos 6y ago
> glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Anothre reason is that glibc is equally bad for dynamic linking. Often a binary compiled on recent glibc doesn't work with older glibc. It is certainly possible but quite challenging to compile portable executables on Linux.

tediousdemise 6y ago

Maybe I hyperbolized that a bit. I think there are pros and cons to each approach. The uses cases for static vs dynamic are also different. There are compile time disadvantages to static libraries, and there are security disadvantages to shared libraries. But calling one method better than the other is inane.

Arnt 6y ago
Are you aware of the runtime differences? Both time and space. Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. Compiling with -fPIC reserves one register, which isn't a problem unless you're on a register-starved CPU, which is all of them.
- zozbot234 6y ago
  > Shared libraries (at least with ELF) requires fixups, which involve COW and a considerable number of per-process pages. You can avoid load-time fixups via prelinking.
- floatboth 6y ago
  > register-starved CPU, which is all of them Really? None of the 64 bit ISAs seem particularly starved, especially not aarch64.
  - kelnos 6y ago
    I think the parent's point is that all registers are precious, and freeing up even one by avoiding dynamic might be worth doing.
  - Arnt 6y ago
    It's not particularly short of registers, but IIRC even aarch64 is short enough that they implemented register renaming. I agree that taking a register away is a much less pressing issue there than on the x86.
    - CalChris 6y ago
      The purpose of register renaming in the AArch64 microarchitectures is to support out-of-order. By comparison, you don't find register renaming in the in-order A53. https://en.wikichip.org/wiki/arm_holdings/microarchitectures/cortex-a53 https://en.wikichip.org/wiki/arm_holdings/microarchitectures...
      - Arnt 6y ago
        Suppose you had enough registers, ie. so many that there's no real reason to compilers to save. In that case, you could "support" out-of-order execution by adding a couple sentences to the compiler guidance documentation for your CPU, telling them that they'll get better performance on o-o-o cores if they wait as long as practical with reusing registers.
        CalChris 6y ago
        They've tried variations of this. It takes instruction bits to describe these registers: 32 regs is 5 bits, target + src1 + src2 is 15 bits already. They've tried scratch pads. The Mill CPU's Belt is yet another approach. https://en.wikipedia.org/wiki/Scratchpad_memory https://millcomputing.com/ How to Use 1000 Registers https://caltechconf.library.caltech.edu/200/ 1000 registers would be 10 bits. So they tried register windows. These are all good ideas but they get to compete. So far, I'm a huge AArch64 fan but I now see the purpose of RISC-V. I watched Chris Lattner's ASPLOS talk and it finally clicked. https://www.youtube.com/watch?v=4HgShra-KnY

AnthonyMouse 6y ago

> Considering the popularity of vendoring and/or static linking dependencies, I'm not sure where you're getting the "almost no one else agrees with" from. Vendoring is hugely popular for projects in year zero, because it makes it easier to ship. Until they learn that packaging specific versions of all your dependencies with your project implies taking on responsibility for backporting all security updates to that version of every dependency when upstream is only actively maintaining the latest version. Or you have a project riddled with known published security vulnerabilities, which is what most of those turn into.

lanstin 6y ago
You have two choices here. Keep updating as you go along and amortize the price for so much open source deps inability to avoid breaking change; or don’t do that and then eight years later lead a heroic update the deps project that takes ten people and millions of dollars and some availability hits. You keep the principal of change only one thing at a time the first way and get promotions the second way. Static vs dynamic is just do you want the code owners to be responsible or some centralized ops group to be responsible. Usually ops doesn’t have the insight to do a good job with deps, so it’s better to have dev do it, e.g. statically linked go binaries, statically linked C/c++, frozen package versions in Python, I think mvn has a similar capability, also version pinned base container images etc etc. hence the fact all the packaging systems add new versions and only rarely remove things as that breaks it all worse than a security hole. As a dev person with a lot of ops experience I like static because I don’t want some janky dependency change to come into the execution path when I am not aware of making a change. On the other hand, people usually just freeze everything and then don’t worry about it, which is wrong. But your security team needs to review source and source deps not just deployed versions. So if code is still owned, the bad deps can be found and fixed either way.
- lanstin 6y ago
  I think a case for dynamic loading of your SSL library can be made.
hibbelig 6y ago

attractivechaos 6y ago

> glibc's hostility towards static linking is arguably a substantial part of why musl has attracted so much interest. Anothre reason is that glibc is equally bad for dynamic linking. Often a binary compiled on recent glibc doesn't work with older glibc. It is certainly possible but quite challenging to compile portable executables on Linux.