3 ms·

Kernel samepage merging and zram/zswap are the answers to the memory usage issue. The only actual issues with static linking is that it's harder to ensure that

Kernel samepage merging and zram/zswap are the answers to the memory usage issue. The only actual issues with static linking is that it's harder to ensure that bugs and security vulnerabilites get patched.

yjftsjthsd-h 2y ago
I disagree: - KSM only works on pages that are identical; I am skeptical that it can actually identify 2 instances of a lib after it's been through an optimizing compiler+linker (mostly LTO) - KSM has performance overhead - zram/zswap only let you reduce the impact of running out of memory at the cost of performance; you're still swapping, no matter how much we improve the details, so it'll always be worse than outright deduplicating libraries
- MrDrMcCoy 2y ago
  Fair points overall. I think KSM is still useful and worth the overhead for things like sprawling browser tabs and densely deployed microservices. I disagree that swap is only (or even primarily) for situations where you're running out of memory, though. It is useful for reducing I/O contention and improving memory utilization by replacing underutilized pages with file cache when it makes sense to do so. See this for more details: https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
  - yjftsjthsd-h 2y ago
    Microservices quite likely, but I'd expect browsers to get ~zero benefit from KSM; aren't browsers all thin-copy forked from a single process anyways? And yes I sort of agree with swapping unused stuff so you have more room for cache, but that's only good for unused stuff; if you swap the programs you're actually running - even just zram swap - you're going to hurt performance.
pizlonator 2y ago
Same page merging is not going to work with static linking. The same modules linked into two different programs will likely have zero identical pages. Zram/zswap isnt nearly as good as the kernel knowing straight from the outset that all processes want exactly the same mappings from the same files.

yjftsjthsd-h 2y ago

I disagree: - KSM only works on pages that are identical; I am skeptical that it can actually identify 2 instances of a lib after it's been through an optimizing compiler+linker (mostly LTO) - KSM has performance overhead - zram/zswap only let you reduce the impact of running out of memory at the cost of performance; you're still swapping, no matter how much we improve the details, so it'll always be worse than outright deduplicating libraries

MrDrMcCoy 2y ago
Fair points overall. I think KSM is still useful and worth the overhead for things like sprawling browser tabs and densely deployed microservices. I disagree that swap is only (or even primarily) for situations where you're running out of memory, though. It is useful for reducing I/O contention and improving memory utilization by replacing underutilized pages with file cache when it makes sense to do so. See this for more details: https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
- yjftsjthsd-h 2y ago
  Microservices quite likely, but I'd expect browsers to get ~zero benefit from KSM; aren't browsers all thin-copy forked from a single process anyways? And yes I sort of agree with swapping unused stuff so you have more room for cache, but that's only good for unused stuff; if you swap the programs you're actually running - even just zram swap - you're going to hurt performance.

pizlonator 2y ago

Same page merging is not going to work with static linking. The same modules linked into two different programs will likely have zero identical pages. Zram/zswap isnt nearly as good as the kernel knowing straight from the outset that all processes want exactly the same mappings from the same files.