5 ms·

That's also a good argument for shared libraries, but the memory usage argument is a big one. I have 583 processes running on my Linux box right now (I'm postin

That's also a good argument for shared libraries, but the memory usage argument is a big one. I have 583 processes running on my Linux box right now (I'm posting from FF running on GNOME on Pop!_OS), and it's nice that when they run the same code (like zlib but also lots of things that are bigger than zlib), they load the same shared library, which means they are using the same mapped file and so the physical memory is shared. It means that memory usage due to code scales with the amount of code (linear), not with the amount of processes times the amount of code (quadratic). I think that having a sophisticated desktop OS where every process had distinct physical memory for duplicated uses of the same code would be problematic at scale, especially on systems with less RAM. At least that physical memory would still be disk-backed, but that only goes so far. That said, the security argument is also a good argument!

MrDrMcCoy 2y ago
Kernel samepage merging and zram/zswap are the answers to the memory usage issue. The only actual issues with static linking is that it's harder to ensure that bugs and security vulnerabilites get patched.
- yjftsjthsd-h 2y ago
  I disagree: - KSM only works on pages that are identical; I am skeptical that it can actually identify 2 instances of a lib after it's been through an optimizing compiler+linker (mostly LTO) - KSM has performance overhead - zram/zswap only let you reduce the impact of running out of memory at the cost of performance; you're still swapping, no matter how much we improve the details, so it'll always be worse than outright deduplicating libraries
  - MrDrMcCoy 2y ago
    Fair points overall. I think KSM is still useful and worth the overhead for things like sprawling browser tabs and densely deployed microservices. I disagree that swap is only (or even primarily) for situations where you're running out of memory, though. It is useful for reducing I/O contention and improving memory utilization by replacing underutilized pages with file cache when it makes sense to do so. See this for more details: https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
    - yjftsjthsd-h 2y ago
      Microservices quite likely, but I'd expect browsers to get ~zero benefit from KSM; aren't browsers all thin-copy forked from a single process anyways? And yes I sort of agree with swapping unused stuff so you have more room for cache, but that's only good for unused stuff; if you swap the programs you're actually running - even just zram swap - you're going to hurt performance.
- pizlonator 2y ago
  Same page merging is not going to work with static linking. The same modules linked into two different programs will likely have zero identical pages. Zram/zswap isnt nearly as good as the kernel knowing straight from the outset that all processes want exactly the same mappings from the same files.

MrDrMcCoy 2y ago

Kernel samepage merging and zram/zswap are the answers to the memory usage issue. The only actual issues with static linking is that it's harder to ensure that bugs and security vulnerabilites get patched.

yjftsjthsd-h 2y ago
I disagree: - KSM only works on pages that are identical; I am skeptical that it can actually identify 2 instances of a lib after it's been through an optimizing compiler+linker (mostly LTO) - KSM has performance overhead - zram/zswap only let you reduce the impact of running out of memory at the cost of performance; you're still swapping, no matter how much we improve the details, so it'll always be worse than outright deduplicating libraries
- MrDrMcCoy 2y ago
  Fair points overall. I think KSM is still useful and worth the overhead for things like sprawling browser tabs and densely deployed microservices. I disagree that swap is only (or even primarily) for situations where you're running out of memory, though. It is useful for reducing I/O contention and improving memory utilization by replacing underutilized pages with file cache when it makes sense to do so. See this for more details: https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
  - yjftsjthsd-h 2y ago
    Microservices quite likely, but I'd expect browsers to get ~zero benefit from KSM; aren't browsers all thin-copy forked from a single process anyways? And yes I sort of agree with swapping unused stuff so you have more room for cache, but that's only good for unused stuff; if you swap the programs you're actually running - even just zram swap - you're going to hurt performance.
pizlonator 2y ago
Same page merging is not going to work with static linking. The same modules linked into two different programs will likely have zero identical pages. Zram/zswap isnt nearly as good as the kernel knowing straight from the outset that all processes want exactly the same mappings from the same files.