5 ms·

> "but it can swap out unused areas" in which case you should fix the program to not allocate a ton of memory it doesn't need Isn't this basically asking every

• morelisp 4y ago
  > Isn't this basically asking every application to independently reimplement their own swap-style functionality? No, it's asking for applications not to allocate a bunch of space they don't need in the first place.
  • shawnz 4y ago
    If you are suggesting that your applications are allocating a bunch of memory which they literally never use in any way, then I don't think that is really a common issue in practice. In practice I expect the typical scenario you're thinking of is applications are allocating a bunch of memory which they use only briefly or rarely and could easily fetch or recompute that data again later if needed. That's exactly the scenario which swap optimizes for. So why should every application individually implement logic to optimize for it? Fixing these kind of issues is not as simple as "just don't allocate the memory".
    • morelisp 4y ago
      > If you are suggesting that your applications are allocating a bunch of memory which they literally never use in any way, then I don't think that is really a common issue in practice. That is what I'm suggesting, and I suggest you look at how much space is wasted by startup-initialized data in libraries for features you'll never use, JIT representations that never get compiled for more than one shape, class metadata that never gets touched and vtables which don't get used, how easy it is in any GC language to accidentally keep a large buffer alive until the end of a lexical scope rather than its last real use, etc. etc. Today's programs are bloated as hell dude.
      • shawnz 4y ago
        All these problems you're describing are way more complicated than "just don't allocate the memory" and would involve sophisticated logic to optimize (like predicting exactly what features are or are not going to be used by an application at start time without significantly increasing the startup latency, etc). Why not just let swap be that optimization?
        • morelisp 4y ago
          Mediocrity uber alles forever!
          • shawnz 4y ago
            I'd argue that the fact such a simple and elegant solution as virtual memory is able to effectively cover such a wide range of use cases while eliminating the need for every application developer to duplicate their efforts is the exact opposite of "mediocre" and in fact it's the exact kind of efficiency-minded thinking that the software industry needs more of.
        • lazide 4y ago
          Because it’s dangerous. Are you SURE that memory never gets accessed under load? Because I’ve seen a lot of surprised sysadmins over the years.
          • shawnz 4y ago
            So why not address those edge cases with something like "madvise" style configuration rather than just saying application developers need to reinvent the wheel any time they do something outside the golden path?
            • lazide 4y ago
              Because that would require the programmer do something explicit to deal with a runtime edgecase they fundamentally don’t have the context to decide the right course of action on? Especially if it’s a library or the like? That infrequent call might be key for program stability in one context or pointless (garbage collector whatever- key in a major server program, pointless in a toy app). The moral of the story is, if under load code or data gets moved from where access is fast (RAM) to where it is slow ( even SSD and most NVMe counts here) then gets accessed when under load? It makes the problem much worse.
    • lazide 4y ago
      Actually no - this is the situation that swap kills your system. That rarely used code path will now take 50 seconds because it got called when it was swapped out (and the system is under memory pressure), instead of either OOM’ing awhile ago or completing in a couple microseconds like it usually does. And ‘infrequently called’ here could be every couple minutes.
      • Dylan16807 4y ago
        If the system is under such strong memory pressure despite swap then you already screwed up. And trying to run that much without swap would be even worse.
        • morelisp 4y ago
          > And trying to run that much without swap would be even worse. No, swap makes it worse. In virtually all cases I’d rather die than stall or flap.
          • Dylan16807 4y ago
            Having no swap won't prevent stalling out. Systems can do that just fine by evicting necessary executable code over and over.
            • lazide 4y ago
              Only in a very, very tiny window. I’ve never seen a system sustain it for more than a trivial amount of time before it OOM kills something and then voila, enough memory. I’m sure someone here will chime in with their example though.
              • Dylan16807 4y ago
                I mean, it happened to me twice in the last couple months on a laptop where I did a lazy install and didn't set up swap. Firefox and 2-3 smaller programs ate up so much memory that it stopped responding for multiple minutes and then went completely dead. I rarely need or set up much swap on a server, but I've managed to have the same kind of problem with the OOM killer not kicking in very quickly on a server. And on my desktops swap is able to soak up many gigabytes and improve my performance a lot with thrashing basically never happening.