6 ms·
The basic argument is that swap can be preferrable to hard eviction for managing disk cache, which can be true, but disk-heavy workloads are less common than ev
by morelisp 4y ago
The basic argument is that swap can be preferrable to hard eviction for managing disk cache, which can be true, but disk-heavy workloads are less common than ever especially in general-use clusters.
(The other argument is the usual mix of "but it can swap out unused areas" in which case you should fix the program to not allocate a ton of memory it doesn't need; and "but the OOM killer will come" and yes that's the entire point please move my process elsewhere).
- shawnz 4y ago> "but it can swap out unused areas" in which case you should fix the program to not allocate a ton of memory it doesn't need Isn't this basically asking every application to independently reimplement their own swap-style functionality? Surely it makes more sense to have swap as a system level feature that every application can take advantage of? That way the system has more information available to make swapping decisions since it can take into account the memory usage of all applications and drivers/etc together, and applications can take advantage of the most advanced swapping algorithms in the latest versions of the system without having to be individually updated.
- morelisp 4y ago> Isn't this basically asking every application to independently reimplement their own swap-style functionality? No, it's asking for applications not to allocate a bunch of space they don't need in the first place.
- shawnz 4y agoIf you are suggesting that your applications are allocating a bunch of memory which they literally never use in any way, then I don't think that is really a common issue in practice. In practice I expect the typical scenario you're thinking of is applications are allocating a bunch of memory which they use only briefly or rarely and could easily fetch or recompute that data again later if needed. That's exactly the scenario which swap optimizes for. So why should every application individually implement logic to optimize for it? Fixing these kind of issues is not as simple as "just don't allocate the memory".
- morelisp 4y ago> If you are suggesting that your applications are allocating a bunch of memory which they literally never use in any way, then I don't think that is really a common issue in practice. That is what I'm suggesting, and I suggest you look at how much space is wasted by startup-initialized data in libraries for features you'll never use, JIT representations that never get compiled for more than one shape, class metadata that never gets touched and vtables which don't get used, how easy it is in any GC language to accidentally keep a large buffer alive until the end of a lexical scope rather than its last real use, etc. etc. Today's programs are bloated as hell dude.
- shawnz 4y agoAll these problems you're describing are way more complicated than "just don't allocate the memory" and would involve sophisticated logic to optimize (like predicting exactly what features are or are not going to be used by an application at start time without significantly increasing the startup latency, etc). Why not just let swap be that optimization?
- morelisp 4y agoMediocrity uber alles forever!
- shawnz 4y agoI'd argue that the fact such a simple and elegant solution as virtual memory is able to effectively cover such a wide range of use cases while eliminating the need for every application developer to duplicate their efforts is the exact opposite of "mediocre" and in fact it's the exact kind of efficiency-minded thinking that the software industry needs more of.
- lazide 4y agoBecause it’s dangerous. Are you SURE that memory never gets accessed under load? Because I’ve seen a lot of surprised sysadmins over the years.