4 ms·

All this is moot for me since I only use systems with overcommit enabled. Is there a way to disable it for a single program or cgroup, to enable it to deal wit

by remram 5y ago

All this is moot for me since I only use systems with overcommit enabled. Is there a way to disable it for a single program or cgroup, to enable it to deal with out-of-memory conditions? Maybe changing/hooking the standard library? mlockall(2) seems overkill, since it will also force all code and mapped files to be resident.

wyldfire 5y ago
You can probably use a pool allocator that won't go back to the OS for more when it's exhausted. I wouldn't be surprised if the popular allocators - jemalloc/tcmalloc/scudo or even the defaults in glibc/musl support a setting like this.
- remram 5y ago
  I suppose I could use a custom allocator that calls mlock() on each page. I will look whether the ones you mention have Rust integrations and whether they expose a setting like that.
  - eqvinox 5y ago
    I understood wyldfire's post to mean counting & imposing your own artificial memory limit, no mlock involved. But I wouldn't know how to pick that limit :/.
    - wyldfire 5y ago
      Yes that is what I meant. And it would avoid the latency hit of a system call to satisfy an allocation. However doing an mlock would avoid the latency hit of paging, so that's good too.
    - remram 5y ago
      Oh I see. Well I'm not trying to add a limit to my RSS, I just want to be notified when allocation fail in my process on my machine where overcommit is enabled. Maybe what I'm asking for makes no sense though, because even if my process handles out of memory errors gracefully, it might still get OOM-killed when another process allocates some more.
hyperman1 5y ago
Can't you use ulimit for this? Finding out the exact flag combination might be gnarly
- remram 5y ago
  I don't think so. You can set a resident size limit, but you can't force all anonymous memory to be resident (or non-overcommitted).
  - hyperman1 5y ago
    I was thinking about -v, maximum virtual memory size https://ss64.com/bash/ulimit.html https://ss64.com/bash/ulimit.html
    - eqvinox 5y ago
      In a way, it's counterproductive to limit VSZ. It's perfectly fine to mmap() gigabytes of files to virtual addresses, but you neither expect nor want to have that counted against your "memory usage" limit. The entire problem is, in a way, unsolveable with current OS APIs. AFAIK there is no preexisting, good, actually usable, and universal memory usage meter. Some things work for a lot of cases, but I don't think there's anything that would work universally. Coincidentally, in my eyes the best way to handle memory pressure in applications would be proactively rather than reactively. Sometimes you can unload more things, like pages in a document that aren't being edited right now. And if you need to fail, you can fail on good boundaries (e.g. refusing entire requests in server-like things) rather than in the middle of something that you need tons of work to unwind correctly. (Slightly related: pool memory allocators, e.g. APR's https://apr.apache.org/docs/apr/trunk/group__apr__pools.html https://apr.apache.org/docs/apr/trunk/group__apr__pools.html )
eqvinox 5y ago
Indeed it's pointless for desktop/server OSes. But at the same time it's critical for kernel and embedded (µC, the "too small for Linux" kind) code.
- Narishma 5y ago
  > Indeed it's pointless for desktop/server OSes. Isn't it only Linux that does overcommit by default?
pornel 5y ago
It's bonkers to allow programs to destabilize the whole machine to the point that the kernel has to start killing off processes just to survive. I use https://lib.rs/cap https://lib.rs/cap which self-imposes memory limit on the process. Setting that limit below cgroup limit allows programs to actually handle OOM before they get killed by the OS (if only Rust's libstd wasn't so eager to self-abort anyway).
- creata 5y ago
  > I use https://lib.rs/cap https://lib.rs/cap which self-imposes memory limit on the process. How do you choose that limit?
  - pornel 5y ago
    In case of servers/vm/containers you probably know how much RAM you've provisioned, so set it to that minus whatever other processes need to live. For desktop applications it's tough. It may be just an arbitrarily high amount you don't expect to hit during normal operation. If you need to work with variable-size data, then it could be `size_of_file_being_opened * x` if you can predict the `x`.

by remram 5y ago

wyldfire 5y ago

You can probably use a pool allocator that won't go back to the OS for more when it's exhausted. I wouldn't be surprised if the popular allocators - jemalloc/tcmalloc/scudo or even the defaults in glibc/musl support a setting like this.

remram 5y ago
I suppose I could use a custom allocator that calls mlock() on each page. I will look whether the ones you mention have Rust integrations and whether they expose a setting like that.
- eqvinox 5y ago
  I understood wyldfire's post to mean counting & imposing your own artificial memory limit, no mlock involved. But I wouldn't know how to pick that limit :/.
  - wyldfire 5y ago
    Yes that is what I meant. And it would avoid the latency hit of a system call to satisfy an allocation. However doing an mlock would avoid the latency hit of paging, so that's good too.
  - remram 5y ago
    Oh I see. Well I'm not trying to add a limit to my RSS, I just want to be notified when allocation fail in my process on my machine where overcommit is enabled. Maybe what I'm asking for makes no sense though, because even if my process handles out of memory errors gracefully, it might still get OOM-killed when another process allocates some more.

hyperman1 5y ago

Can't you use ulimit for this? Finding out the exact flag combination might be gnarly

remram 5y ago
I don't think so. You can set a resident size limit, but you can't force all anonymous memory to be resident (or non-overcommitted).
- hyperman1 5y ago
  I was thinking about -v, maximum virtual memory size https://ss64.com/bash/ulimit.html https://ss64.com/bash/ulimit.html
  - eqvinox 5y ago
    In a way, it's counterproductive to limit VSZ. It's perfectly fine to mmap() gigabytes of files to virtual addresses, but you neither expect nor want to have that counted against your "memory usage" limit. The entire problem is, in a way, unsolveable with current OS APIs. AFAIK there is no preexisting, good, actually usable, and universal memory usage meter. Some things work for a lot of cases, but I don't think there's anything that would work universally. Coincidentally, in my eyes the best way to handle memory pressure in applications would be proactively rather than reactively. Sometimes you can unload more things, like pages in a document that aren't being edited right now. And if you need to fail, you can fail on good boundaries (e.g. refusing entire requests in server-like things) rather than in the middle of something that you need tons of work to unwind correctly. (Slightly related: pool memory allocators, e.g. APR's https://apr.apache.org/docs/apr/trunk/group__apr__pools.html https://apr.apache.org/docs/apr/trunk/group__apr__pools.html )

eqvinox 5y ago

Indeed it's pointless for desktop/server OSes. But at the same time it's critical for kernel and embedded (µC, the "too small for Linux" kind) code.

Narishma 5y ago
> Indeed it's pointless for desktop/server OSes. Isn't it only Linux that does overcommit by default?

pornel 5y ago

It's bonkers to allow programs to destabilize the whole machine to the point that the kernel has to start killing off processes just to survive. I use https://lib.rs/cap https://lib.rs/cap which self-imposes memory limit on the process. Setting that limit below cgroup limit allows programs to actually handle OOM before they get killed by the OS (if only Rust's libstd wasn't so eager to self-abort anyway).

creata 5y ago
> I use https://lib.rs/cap https://lib.rs/cap which self-imposes memory limit on the process. How do you choose that limit?
- pornel 5y ago
  In case of servers/vm/containers you probably know how much RAM you've provisioned, so set it to that minus whatever other processes need to live. For desktop applications it's tough. It may be just an arbitrarily high amount you don't expect to hit during normal operation. If you need to work with variable-size data, then it could be `size_of_file_being_opened * x` if you can predict the `x`.