4 ms·

Introduce DAMON-based Proactive Reclamation

32 points by marcodiego 5y ago

marcodiego 5y ago
Linux behavior under memory pressure on the desktop has been a problem for a long time[0]. Although not aimed at the desktop, I hope this combined with zram, cleancache, frontswap and all the infrastructure required to support systemd-oomd finally fixes this. [0] https://lkml.org/lkml/2019/8/4/15 https://lkml.org/lkml/2019/8/4/15
- post-factum 5y ago
  What's described there is disk thrashing, and while there might be different approaches on fixing it, I'd suggest checking [1]. [1] https://gitlab.com/post-factum/pf-kernel/-/commit/1102c90c79d4c2bc5f5a593a94da7e34c65cfda7 https://gitlab.com/post-factum/pf-kernel/-/commit/1102c90c79...
The_rationalist 5y ago
I wonder how does this compare to the state of the art solutions from the author of Nohang: https://github.com/hakavlad/prelockd https://github.com/hakavlad/prelockd https://github.com/hakavlad/memavaild https://github.com/hakavlad/memavaild https://github.com/hakavlad/le9-patch https://github.com/hakavlad/le9-patch in terms of % saved memory or % prevented hangs/OOM / lost base runtime performance
- marcodiego 5y ago
  An interesting possibility would be for, once backing store is full, if any new page should go to swap, the LRU page on the backing store should be "forgotten" and freed; if a "forgotten" page was ever used again, such process would receive the SIGBUS signal. That would keep the system running happily even if processes leak arbitrary amounts of memory.
geekuillaume 5y ago
Phoronix made a small article about this: https://www.phoronix.com/scan.php?page=news_item&px=DAMON-Page-Reclamation-RFC https://www.phoronix.com/scan.php?page=news_item&px=DAMON-Pa...
c7DJTLrn 5y ago
This seems like a massively overcomplicated solution to a simple problem. If you're running out of memory, it's because: 1. You don't have enough memory for what you're trying to do 2. Your software has a memory leak Add more memory to the system or fix your software. Why is this fancy feature needed at all? It's simpler and safer just to reboot and have a fresh start if you're out of memory. Set panic_on_oom, job done.
- KronisLV 5y ago
  Hmm, while i definitely agree with 2., i'm not sure that i'd dismiss 1. so easily. To me, that sounds a bit like saying: "Compressing backups? If you can't store them as they are, you don't have enough storage for what you're trying to do." With finite resources, regardless of whether it's storage, RAM or CPU cores, it makes sense to look for options to utilize them more efficiently. Not everyone can acquire all of the RAM for the problems that they'd want to tackle and therefore solutions like this could provide tangible benefits with sometimes acceptable drawbacks (the mentioned CPU usage). Personally, i feel like having VPSes that'd offer me more memory would be a good thing, since most of the software that i use is mostly memory not CPU constrained, though i understand why opinions could differ. As for memory leaks and rather liberal memory usage - i feel that it'll be inevitable for as long as the industry uses Java, .NET, Ruby, Python, Node and most other technologies with a high abstraction level (and VM runtimes, ofen coupled with GC). Yet not everyone is keen on writing their business apps in C++ or Rust.
  - mananaysiempre 5y ago
    FWIW, there has been a lot of research on making GC (particularly on the JVM) communicate its memory status to the kernel, but the (usually Linux) kernel developers’ attitude was always “no, you don’t get to put your thesis into mainline, unless you can prove people actually use it”. Which is partly fair, but the end result is still a dumb chicken-and-egg situation.
- pantalaimon 5y ago
  > Set panic_on_oom, job done. So any buggy app (or just a `make -j`) can crash the system? Yea no thanks

32 points by marcodiego 5y ago

Linux behavior under memory pressure on the desktop has been a problem for a long time[0]. Although not aimed at the desktop, I hope this combined with zram, cleancache, frontswap and all the infrastructure required to support systemd-oomd finally fixes this. [0] https://lkml.org/lkml/2019/8/4/15 https://lkml.org/lkml/2019/8/4/15

post-factum 5y ago
What's described there is disk thrashing, and while there might be different approaches on fixing it, I'd suggest checking [1]. [1] https://gitlab.com/post-factum/pf-kernel/-/commit/1102c90c79d4c2bc5f5a593a94da7e34c65cfda7 https://gitlab.com/post-factum/pf-kernel/-/commit/1102c90c79...

The_rationalist 5y ago

I wonder how does this compare to the state of the art solutions from the author of Nohang: https://github.com/hakavlad/prelockd https://github.com/hakavlad/prelockd https://github.com/hakavlad/memavaild https://github.com/hakavlad/memavaild https://github.com/hakavlad/le9-patch https://github.com/hakavlad/le9-patch in terms of % saved memory or % prevented hangs/OOM / lost base runtime performance

marcodiego 5y ago
An interesting possibility would be for, once backing store is full, if any new page should go to swap, the LRU page on the backing store should be "forgotten" and freed; if a "forgotten" page was ever used again, such process would receive the SIGBUS signal. That would keep the system running happily even if processes leak arbitrary amounts of memory.

geekuillaume 5y ago

Phoronix made a small article about this: https://www.phoronix.com/scan.php?page=news_item&px=DAMON-Page-Reclamation-RFC https://www.phoronix.com/scan.php?page=news_item&px=DAMON-Pa...

c7DJTLrn 5y ago

This seems like a massively overcomplicated solution to a simple problem. If you're running out of memory, it's because: 1. You don't have enough memory for what you're trying to do 2. Your software has a memory leak Add more memory to the system or fix your software. Why is this fancy feature needed at all? It's simpler and safer just to reboot and have a fresh start if you're out of memory. Set panic_on_oom, job done.

KronisLV 5y ago
Hmm, while i definitely agree with 2., i'm not sure that i'd dismiss 1. so easily. To me, that sounds a bit like saying: "Compressing backups? If you can't store them as they are, you don't have enough storage for what you're trying to do." With finite resources, regardless of whether it's storage, RAM or CPU cores, it makes sense to look for options to utilize them more efficiently. Not everyone can acquire all of the RAM for the problems that they'd want to tackle and therefore solutions like this could provide tangible benefits with sometimes acceptable drawbacks (the mentioned CPU usage). Personally, i feel like having VPSes that'd offer me more memory would be a good thing, since most of the software that i use is mostly memory not CPU constrained, though i understand why opinions could differ. As for memory leaks and rather liberal memory usage - i feel that it'll be inevitable for as long as the industry uses Java, .NET, Ruby, Python, Node and most other technologies with a high abstraction level (and VM runtimes, ofen coupled with GC). Yet not everyone is keen on writing their business apps in C++ or Rust.
- mananaysiempre 5y ago
  FWIW, there has been a lot of research on making GC (particularly on the JVM) communicate its memory status to the kernel, but the (usually Linux) kernel developers’ attitude was always “no, you don’t get to put your thesis into mainline, unless you can prove people actually use it”. Which is partly fair, but the end result is still a dumb chicken-and-egg situation.
pantalaimon 5y ago
> Set panic_on_oom, job done. So any buggy app (or just a `make -j`) can crash the system? Yea no thanks