11 ms·

Dealing with Out-of-Memory Conditions in Rust

118 points by zbentley 5y ago

monocasa 5y ago
FWIW, this is a little outdated now. If you're willing to bite off nightly, there's cfg(no_global_oom_handling) which will just remove access to the aborting allocation API. It came out of the Linux kernel work and their very similar concerns IIRC.
- tialaramex 5y ago
  If you cfg(no_global_oom_handling) lots of nice things go away. Which is both appropriate - those nice things did in fact depend on global OOM handling - and hopefully likely to keep more people from erroneously believing they can't afford global OOM handling. As an example, ordinarily on Rust this does what it looks like it does: let mut lyrics = "Never was".to_owned(); lyrics += " a cornflake girl"; With cfg(no_global_oom_handling) this can't work, most importantly because the second line is an arbitrary concatenation and therefore allocates, but there is no possible way to signal if it fails.
  - AlexanderDhoore 5y ago
    As an embedded C guy this makes me so happy. Yes, that second line allocates and the fact that it is hidden makes me nervous. I want a programming environment that exposes all of the complexity and makes me deal with the consequences.
    - tialaramex 5y ago
      Linus feels similarly, which is why this feature was mandatory to land Rust-for-Linux and why C++ in Linux was never likely to go anywhere.
  - Waterluvian 5y ago
    Newbie question: How hard would it be for a text editor to highlight all lines that do or might allocate? Can they be known statically?
    - wizzwizz4 5y ago
      rust-analyzer could probably do it. That information isn't exposed in type signatures, though, so it'd have to have full access to the source of everything and a good understanding of the standard library (and would fail if you added your own “might allocate” by going around the back of the global allocator).
    - 1f60c 5y ago
      I'm pretty sure catching this in all cases would involve solving the Halting Problem.
      - Waterluvian 5y ago
        I think if we wanted a concrete answer you’d be right. But I feel like a static analyzer could clearly identify “this line can allocate” just by knowing what language features can allocate. And for third party library methods “does this function have anything inside of it that could allocate?”
      - oconnor663 5y ago
        If you're allowed to have false positives, it's not so hard. Like "this function allocates if A or B is true" just becomes "this function can allocate". It's not so different from finding functions that can panic.
      - Xylakant 5y ago
        I’m fairly certain that this does not require solving the halting problem. Fundamentally, every line of code can be transformed to the underlying code constructs, and those could be checked for allocations. Doing this in an efficient manner is probably hard, but certainly not impossible
        5y ago
  - umanwizard 5y ago
    I’ve been doing Rust full-time for two years and I had no idea you could do this. I’d just have used `push_str` in any case. Giving this up doesn’t seem very impactful and I’d be happy to do so to enable actually useful features.
    - tialaramex 5y ago
      You can't push_str() under cfg(no_global_oom_handling) either I mean, they did warn you: No Global OOM Handling. So, if this mustn't fail, and it might not succeed, therefore it has to be eliminated, a String under cfg(no_global_oom_handling) can't push_str() and it can't reserve() and it can't do a lot of things. Their definitions are conditionally removed from the type. I didn't check, it's possible "something".to_owned() is similarly forbidden, seems like that would allocate memory. Basically if you live in a world where allocating memory for strings, vectors, and other growable types feels extravagant, cfg(no_global_oom_handling) is for you, and if not then maybe you should re-evaluate why you are worried about allocation failures when you are wasting precious heap memory on such data structures.
      - umanwizard 5y ago
        > You can't push_str() under cfg(no_global_oom_handling) either Thanks for the clarification. I had totally missed the point.
superjan 5y ago
Why not budget how much memory your rust component needs, and allocate a pool ahead of time? This is how it’s sometimes done in embedded systems. It depends on what your rust code does of course, but if it’s runtime memory use depends on untrusted data it is handling that is a concern in itself.
- develatio 5y ago
  I'm not sure if this can apply to the most commonly used software. A browser can't know in advance how much memory it would need. Same thing applies for audio, image or video editing software. Probably office suites too. This might be feasible with some limited-in-features apps, like an audio/video player, an IM client, etc...
  - 1f60c 5y ago
    ...and missiles: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98125 https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...
  - superjan 5y ago
    It won’t. I was thinking more about backend components where you allocate a fixed per request memory budget. It would work best if you could apply the budget per in-flight request. In case a request goes over budget, only the affected (presumably nefarious) request gets terminated, other requests and other processes will run uninterrupted.
  - SiebenHeaven 5y ago
    Most commonly used software such as browsers can abort on OOM, this isn't about those cases at all. In conditions that you do not want to abort, usually you can and would want to allocate pool before hand.
tialaramex 5y ago
"where the C++ pieces can all recover from OOM and the Rust component cannot" I would like to read about their successful test of their C++ pieces using the same (random fail small allocations) strategy. My impression from Herb Sutter is that on the popular C++ Standard Library implementations (for MSVC, Clang, GCC) this actually doesn't work, but of course Herb has a reason to say that - so it'd be interesting to hear from somebody who has a reason to believe the opposite.
- the_mitsuhiko 5y ago
  Handling allocation failures on the STL is also only possible with exceptions.
- oconnor663 5y ago
  I think I know the talk you're referring to. This is a great one: https://youtu.be/ARYP83yNAWk?t=56m56s https://youtu.be/ARYP83yNAWk?t=56m56s
lmilcin 5y ago
My experience with out of memory is that in every single language and environment I worked on in the past, once an application hits that condition, there is very little hope of continuing reliably. So if your aim is to build a reliable system, it is much easier to plan to never get there in the first place. Alternatively, make your application restart after OOM. I would actually prefer the application to just stop immediately without unwinding anything. It makes it much clearer as to what possible states the application could have gotten itself to. Hopefully you already have designed the application to move atomically between known states and have mechanism to handle any operation getting interrupted. If you did it right, handling OOM by having the application drop dead should be viewed as just exercising these mechanisms.
- SigmundA 5y ago
  I am not sure how you can make a user friendly application like that. How about an image viewer that tries to open too large an image, should it just crash when it OOMs? I would much prefer an error dialog and the program continues to run.
  - lmilcin 5y ago
    Users are aware things stop working when memory is exhausted. As a user I am ok with this. I don't let my memory end and would prefer that the application worked reliably while there is still memory. What I don't like is applications trying to pretend nothing happened but then doing some strange shit. I have seen, for example, applications using 100% cpu after they hit OOM. Yes they survived. No, it is not user friendly.
    - SigmundA 5y ago
      Why would users be aware what happened? Did you tell them? This is a very reliable and easy for the user and other developers to understand. try { image = LoadImage(path); } catch(OutOfMemoryException e) { Msgbox("Cannot load image it is too large for available memory"); } That is way more friendly than a program crash and allow the user to try again perhaps with a smaller version of the image because they accidentally picked the high res version or something. Either way you may get 100% CPU as the program crashes and memory gets reclaimed, or the program continues to run and the garbage collector reclaims.
remram 5y ago
All this is moot for me since I only use systems with overcommit enabled. Is there a way to disable it for a single program or cgroup, to enable it to deal with out-of-memory conditions? Maybe changing/hooking the standard library? mlockall(2) seems overkill, since it will also force all code and mapped files to be resident.
- wyldfire 5y ago
  You can probably use a pool allocator that won't go back to the OS for more when it's exhausted. I wouldn't be surprised if the popular allocators - jemalloc/tcmalloc/scudo or even the defaults in glibc/musl support a setting like this.
  - remram 5y ago
    I suppose I could use a custom allocator that calls mlock() on each page. I will look whether the ones you mention have Rust integrations and whether they expose a setting like that.
    - eqvinox 5y ago
      I understood wyldfire's post to mean counting & imposing your own artificial memory limit, no mlock involved. But I wouldn't know how to pick that limit :/.
      - wyldfire 5y ago
        Yes that is what I meant. And it would avoid the latency hit of a system call to satisfy an allocation. However doing an mlock would avoid the latency hit of paging, so that's good too.
      - remram 5y ago
        Oh I see. Well I'm not trying to add a limit to my RSS, I just want to be notified when allocation fail in my process on my machine where overcommit is enabled. Maybe what I'm asking for makes no sense though, because even if my process handles out of memory errors gracefully, it might still get OOM-killed when another process allocates some more.
vzaliva 5y ago
Is it the same issue Linus was pointing out when discussing possible adoption of Rust for Linux kernel?
- loeg 5y ago
  Yes. Kernel mode code needs to gracefully handle OOM conditions.
marcinjachymiak 5y ago
It's good to see that OOM errors can be handled in Rust now, at least in unstable. Zig has a similar approach that is pretty cool. I don't know of any other language that let's you handle it like this https://ziglang.org/documentation/0.8.0/#Heap-Allocation-Failure https://ziglang.org/documentation/0.8.0/#Heap-Allocation-Fai...
- dralley 5y ago
  The zig approach seems really well suited to low-level libraries where you really need to handle these errors (and custom allocators, etc.) but it's kind of annoying for general purpose programs.

118 points by zbentley 5y ago

FWIW, this is a little outdated now. If you're willing to bite off nightly, there's cfg(no_global_oom_handling) which will just remove access to the aborting allocation API. It came out of the Linux kernel work and their very similar concerns IIRC.

tialaramex 5y ago
If you cfg(no_global_oom_handling) lots of nice things go away. Which is both appropriate - those nice things did in fact depend on global OOM handling - and hopefully likely to keep more people from erroneously believing they can't afford global OOM handling. As an example, ordinarily on Rust this does what it looks like it does: let mut lyrics = "Never was".to_owned(); lyrics += " a cornflake girl"; With cfg(no_global_oom_handling) this can't work, most importantly because the second line is an arbitrary concatenation and therefore allocates, but there is no possible way to signal if it fails.
- AlexanderDhoore 5y ago
  As an embedded C guy this makes me so happy. Yes, that second line allocates and the fact that it is hidden makes me nervous. I want a programming environment that exposes all of the complexity and makes me deal with the consequences.
  - tialaramex 5y ago
    Linus feels similarly, which is why this feature was mandatory to land Rust-for-Linux and why C++ in Linux was never likely to go anywhere.
- Waterluvian 5y ago
  Newbie question: How hard would it be for a text editor to highlight all lines that do or might allocate? Can they be known statically?
  - wizzwizz4 5y ago
    rust-analyzer could probably do it. That information isn't exposed in type signatures, though, so it'd have to have full access to the source of everything and a good understanding of the standard library (and would fail if you added your own “might allocate” by going around the back of the global allocator).
  - 1f60c 5y ago
    I'm pretty sure catching this in all cases would involve solving the Halting Problem.
    - Waterluvian 5y ago
      I think if we wanted a concrete answer you’d be right. But I feel like a static analyzer could clearly identify “this line can allocate” just by knowing what language features can allocate. And for third party library methods “does this function have anything inside of it that could allocate?”
    - oconnor663 5y ago
      If you're allowed to have false positives, it's not so hard. Like "this function allocates if A or B is true" just becomes "this function can allocate". It's not so different from finding functions that can panic.
    - Xylakant 5y ago
      I’m fairly certain that this does not require solving the halting problem. Fundamentally, every line of code can be transformed to the underlying code constructs, and those could be checked for allocations. Doing this in an efficient manner is probably hard, but certainly not impossible
      - 5y ago
- umanwizard 5y ago
  I’ve been doing Rust full-time for two years and I had no idea you could do this. I’d just have used `push_str` in any case. Giving this up doesn’t seem very impactful and I’d be happy to do so to enable actually useful features.
  - tialaramex 5y ago
    You can't push_str() under cfg(no_global_oom_handling) either I mean, they did warn you: No Global OOM Handling. So, if this mustn't fail, and it might not succeed, therefore it has to be eliminated, a String under cfg(no_global_oom_handling) can't push_str() and it can't reserve() and it can't do a lot of things. Their definitions are conditionally removed from the type. I didn't check, it's possible "something".to_owned() is similarly forbidden, seems like that would allocate memory. Basically if you live in a world where allocating memory for strings, vectors, and other growable types feels extravagant, cfg(no_global_oom_handling) is for you, and if not then maybe you should re-evaluate why you are worried about allocation failures when you are wasting precious heap memory on such data structures.
    - umanwizard 5y ago
      > You can't push_str() under cfg(no_global_oom_handling) either Thanks for the clarification. I had totally missed the point.

superjan 5y ago

Why not budget how much memory your rust component needs, and allocate a pool ahead of time? This is how it’s sometimes done in embedded systems. It depends on what your rust code does of course, but if it’s runtime memory use depends on untrusted data it is handling that is a concern in itself.

develatio 5y ago
I'm not sure if this can apply to the most commonly used software. A browser can't know in advance how much memory it would need. Same thing applies for audio, image or video editing software. Probably office suites too. This might be feasible with some limited-in-features apps, like an audio/video player, an IM client, etc...
- 1f60c 5y ago
  ...and missiles: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98125 https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...
- superjan 5y ago
  It won’t. I was thinking more about backend components where you allocate a fixed per request memory budget. It would work best if you could apply the budget per in-flight request. In case a request goes over budget, only the affected (presumably nefarious) request gets terminated, other requests and other processes will run uninterrupted.
- SiebenHeaven 5y ago
  Most commonly used software such as browsers can abort on OOM, this isn't about those cases at all. In conditions that you do not want to abort, usually you can and would want to allocate pool before hand.

tialaramex 5y ago

"where the C++ pieces can all recover from OOM and the Rust component cannot" I would like to read about their successful test of their C++ pieces using the same (random fail small allocations) strategy. My impression from Herb Sutter is that on the popular C++ Standard Library implementations (for MSVC, Clang, GCC) this actually doesn't work, but of course Herb has a reason to say that - so it'd be interesting to hear from somebody who has a reason to believe the opposite.

the_mitsuhiko 5y ago
Handling allocation failures on the STL is also only possible with exceptions.
oconnor663 5y ago
I think I know the talk you're referring to. This is a great one: https://youtu.be/ARYP83yNAWk?t=56m56s https://youtu.be/ARYP83yNAWk?t=56m56s

lmilcin 5y ago

My experience with out of memory is that in every single language and environment I worked on in the past, once an application hits that condition, there is very little hope of continuing reliably. So if your aim is to build a reliable system, it is much easier to plan to never get there in the first place. Alternatively, make your application restart after OOM. I would actually prefer the application to just stop immediately without unwinding anything. It makes it much clearer as to what possible states the application could have gotten itself to. Hopefully you already have designed the application to move atomically between known states and have mechanism to handle any operation getting interrupted. If you did it right, handling OOM by having the application drop dead should be viewed as just exercising these mechanisms.

SigmundA 5y ago
I am not sure how you can make a user friendly application like that. How about an image viewer that tries to open too large an image, should it just crash when it OOMs? I would much prefer an error dialog and the program continues to run.
- lmilcin 5y ago
  Users are aware things stop working when memory is exhausted. As a user I am ok with this. I don't let my memory end and would prefer that the application worked reliably while there is still memory. What I don't like is applications trying to pretend nothing happened but then doing some strange shit. I have seen, for example, applications using 100% cpu after they hit OOM. Yes they survived. No, it is not user friendly.
  - SigmundA 5y ago
    Why would users be aware what happened? Did you tell them? This is a very reliable and easy for the user and other developers to understand. try { image = LoadImage(path); } catch(OutOfMemoryException e) { Msgbox("Cannot load image it is too large for available memory"); } That is way more friendly than a program crash and allow the user to try again perhaps with a smaller version of the image because they accidentally picked the high res version or something. Either way you may get 100% CPU as the program crashes and memory gets reclaimed, or the program continues to run and the garbage collector reclaims.

remram 5y ago

All this is moot for me since I only use systems with overcommit enabled. Is there a way to disable it for a single program or cgroup, to enable it to deal with out-of-memory conditions? Maybe changing/hooking the standard library? mlockall(2) seems overkill, since it will also force all code and mapped files to be resident.

wyldfire 5y ago
You can probably use a pool allocator that won't go back to the OS for more when it's exhausted. I wouldn't be surprised if the popular allocators - jemalloc/tcmalloc/scudo or even the defaults in glibc/musl support a setting like this.
- remram 5y ago
  I suppose I could use a custom allocator that calls mlock() on each page. I will look whether the ones you mention have Rust integrations and whether they expose a setting like that.
  - eqvinox 5y ago
    I understood wyldfire's post to mean counting & imposing your own artificial memory limit, no mlock involved. But I wouldn't know how to pick that limit :/.
    - wyldfire 5y ago
      Yes that is what I meant. And it would avoid the latency hit of a system call to satisfy an allocation. However doing an mlock would avoid the latency hit of paging, so that's good too.
    - remram 5y ago
      Oh I see. Well I'm not trying to add a limit to my RSS, I just want to be notified when allocation fail in my process on my machine where overcommit is enabled. Maybe what I'm asking for makes no sense though, because even if my process handles out of memory errors gracefully, it might still get OOM-killed when another process allocates some more.

vzaliva 5y ago

Is it the same issue Linus was pointing out when discussing possible adoption of Rust for Linux kernel?

loeg 5y ago
Yes. Kernel mode code needs to gracefully handle OOM conditions.

marcinjachymiak 5y ago

It's good to see that OOM errors can be handled in Rust now, at least in unstable. Zig has a similar approach that is pretty cool. I don't know of any other language that let's you handle it like this https://ziglang.org/documentation/0.8.0/#Heap-Allocation-Failure https://ziglang.org/documentation/0.8.0/#Heap-Allocation-Fai...

dralley 5y ago
The zig approach seems really well suited to low-level libraries where you really need to handle these errors (and custom allocators, etc.) but it's kind of annoying for general purpose programs.