8 ms·

The problems are that: - most such interfaces do have a constant overhead that all programmers using the language have to pay. - some major operating systems

The problems are that: - most such interfaces do have a constant overhead that all programmers using the language have to pay. - some major operating systems (like Linux, ehem) make these interfaces useless for all their user space apps by enabling overcommit by default, which makes it hard/impossible to write portable code that can handle OOM Explicit OOM handling would mean that most users end up paying a relatively high upfront cost for something that in practice for them (e.g. if they are Linux programmers) delivers no value. For example, on Linux with overcommit (the default), even if the system is out of memory, malloc won't return null. It returns a pointer that's not null, such that your if(ptr == nullptr) will act as if everything is "ok", but then, when you try to read/write that memory, that will trigger a hardware exception, that the kernel will catch, and then the kernel will tell the OOM-killer to "make space" by killing "some process", and maybe your app is just killed, or some other app that your app is working with is killed leading to a race condition, or... or.... When your app is killed by the OOM killer, the signal that your app gets is "irrecoverable", which means that your app will die, you can at best try to do some cleanup before it does, but it will die nevertheless. So I find it extremely ironic for Linus to argue that practical programming languages are hard to use for environments that must handle OOM errors, when they are championing one of the major platforms that makes handling OOM useless by default. I keep saying this, but the obvious fix is for the Linux kernel to use overcommit internally just like they expect user space to do. When the kernel then runs out of memory, it should then start killing drivers at random to make for some space. If they think that's such a great default behavior, they should commit to it. \s

teknopaul 6y ago
You can't solve memory allocation problems by turning off overcommit and handling alloc errors in rust. You don't get Null, you get panic. That's a problem for the kernel, even if you argue it's not a problem elsewhere. Start killing drivers is nuts, oh look my screen went blank, and three HDs stopped working. Damn where is my swap.
- volta83 6y ago
  > Start killing drivers is nuts, I don't disagree, just saying that its as nuts as killing random user-space programs on OOM. "The web browser misbehaved, lets kill the pulseaudio daemon, and if that doesn't fix it, lets kill WiFi, and if that doesn't fix it, lets kill vim, and... " FFS if some app tried to allocate too much memory just let the app know, or kill that app, but don't start randomly killing user-space processes.
  - simiones 6y ago
    While I agree that OOMKiller is pretty mad, it's also important to note that the shared nature of memory means that the app which will die when the system is out of memory is anyway random. Even without overcommit, you can get into the situation of "the web browser is occupying 99% of RAM to open YouTube, and now pulseaudio tried to allocate another 50 bytes that the system just doesn't have, so pulseaudio should handle this, and then the WiFi manager will be the next to need 100 bytes and be refused etc". Even worse, Linux by design stalls to a complete crawl when memory almost runs out, even with 0 swap and a fast disk, as it will first start swapping out code pages before giving up. Which means that every time a process is context switched, its code ends up first needing to be read from disk, the worst possible kind of cache thrashing imaginable.
    - Too 6y ago
      Somehow a LRU-algorithm makes the most sense, if only Linux and humans would have a common definition of what "recently used" means here. And if applications had safe-shutdown and restart on demand mechanisms. On a high level of abstraction this is how a lot of VMs deal with things, such as tabs in your browser or open apps on your smartphone.
nabla9 6y ago
You are talking about userspace. Linus is talking about kernel. You can change the Linux memory overcommit policy using sysctls if you need it.
- teknopaul 6y ago
  die on run out of memory is a bad policy for a userspace server. Seen some hairy production incidents with that, basically solution is piss off customers until they stop coming back. action on run out of memory should be finish up currently running work and hold off new stuff until you have resources, not die and start to reprocess the thundering herd. When you run out of memory last thing you want to do is empty your hot caches.
  - bronson 6y ago
    You’re out of memory. Every possible cache has already been freed. Unless your app is faking it’s own caching, in which case it might be contributing to the problem.
    - angry_octet 6y ago
      It's quite possible that an app is holding onto data that can be flushed, at the expense of future database lookups, session negotiations, etc. Without a proper mechanism in the kernel it is hard to manage memory pressure.
  - nabla9 6y ago
    If that would be important you would disable overcommit in Linux. If you didn't do that, it's probably because it's good tradeoff that saves resource.
pornel 6y ago
The "but the OOM killer!" argument has been a disaster for Rust, and keeps derailing all design discussions. • Linux is not the only OS in the world. • Even on Linux you have containers/cgroups that can impose hard limits. • Platforms without virtual memory are also an important target for Rust. • On 32-bit platforms you can run out of address space before you run out of RAM. • Regardless of what the OS does, the application may still want to impose its own internal limit (e.g. https://lib.rs/cap https://lib.rs/cap) to avoid being OOM-killed or swap death. People keep telling me how Linux never runs out of memory, while I'm currently firefighting a torrent of coredumps caused by Rust's self-own on OOM that actually happens. I generally love Rust, but its OOM handling is awful, and the "but the OOM killer" nonsense is to blame for stalling the absolutely critical fixes it urgently needs.
- teknopaul 6y ago
  word
- varajelle 6y ago
  The overcommit is not the only argument. The second argument is that complex programs that need to handle OOM properly will be full of complex untested error paths. How should one handle a OOM error If one can't allocate some temporary node during an algorithm and how to recover without leaving a corrupted state? That's not a question I want to answer for almost every function call. That's why the rust standard library data structures don't handle OOM. To make it easier to use for the most common case. And for the cases where the OOM handling is required, one can simply use data structures for it, which is what Linux will do (problem solved) Also compare that to the case of mutex poisoning which can return an error on each unlock, and which is now considered a mistake because in practice, nobody want to handle a poisoned mutex error.
  - pornel 6y ago
    I argue it's a rare-enough problem: • Statistically, likelihood of failure is proportional to the allocation size. You'll most often fail to allocate a very large Vec, and then only need to allocate a few bytes for a string that says "error!". I'm using https://lib.rs/fallible_collections https://lib.rs/fallible_collections and just a few strategically placed try_reserve has brought down my coredumps by 99%. • Panic on OOM would unwind, and unwinding tends to free memory rather than allocate anything. The standard library should reserve/preallocate enough memory to be able to start unwinding. • Rust is good at avoiding temporary allocations and relies on the stack a lot. You don't have to use fancy error-handling libraries that collect backtrace on every error. Error handling with enums doesn't touch the heap at all. • Even if everything fails, and you get OOM during OOM handling, that's fair, and still better than abort every time. Note that Rust currently doesn't handle OOM by panicking. It unconditionally aborts the whole process. Libstd could switch to panics without API changes, and that would make OOM handling possible via catch_unwind.
sai_c 6y ago
So, basically you are saying "Rust is mostly used as an application programming language anyways, so let's not pester those users with the overhead of OOM handling"? Fair enough. But this would mean that Rust is (factually) a systems language in the same sense that Go was initially declared to be a systems language. If I look at the Rust community (I give it a try from time to time), I would totally agree with your point, that most users would be pestered by this overhead. I see mostly CLI tools, "web apps" and the kind. Furthermore, I just can't shake off the feeling that the embedded (bare metal, 16 or 32 bit architectures) or (in-house) kernel crowd will always be 2nd class citizens. I keep saying this, but the obvious fix is for the Linux kernel to use overcommit internally just like they expect user space to do. I'm not a kernel developer, but aren't you (maybe) asking for a bit too much? You ask the Linux kernel devs to change the kernel in a significant way, just so you can use Rust for writing drivers in a way you write user space applications.
- dathinab 6y ago
  No it's about "by default" not pestering users. Rust has all the tools needed to gracefully handle memory allocation failure. Yes, the tools are a bit limit wrt. usages in the standard library where you mainly can use them to handle "known to potentially fail big" allocations but not every single small allocation. But enter no_std and it's basically all your choice, which is also the default thing to use for embedded/bar-metal and I count the kernel as embedded/bar-metal. (It's default because the std library types are tuned for the most common use-cases including web-server and user-space system programming, in which you always can have panic on OOM, and error kernel as a much more ergonomic pattern then explicitly returning a result on every thing which potentially allocates). Through without question the discussion around it is a mess.
- volta83 6y ago
  > So, basically you are saying "Rust is mostly used as an application programming language anyways, so let's not pester those users with the overhead of OOM handling"? No. What I am saying is that Rust intended to be a systems programming language that could handle OOM properly everywhere, but that got a lot of opposition because some operating systems, mainly Linux, make it impossible for programs to handle OOM _at all_, so adding proper OOM support to Rust would mean that every Rust Linux programmer would be paying for a feature (in ergonomics, etc.) that they cannot use _at all_. That's the irony. Linux design has made it not worth it for new programming languages to be designed to handle OOM properly, because that's impossible to do in OSes like Linux. People have been complaining about this for user space programs for the last 20 years, and the Linux kernel stand was "that's a feature, not a bug". But now that they are confronted with it, you see Linus writing stuff like "that's not acceptable". That's a huge double standard IMO.
gjulianm 6y ago
> For example, on Linux with overcommit (the default), even if the system is out of memory, malloc won't return null. It returns a pointer that's not null, such that your if(ptr == nullptr) will act as if everything is "ok", Not always. The default overcommit policy will reject certain allocations that are too big. Also, you can change the overcommit policy if you need it. Also, overcommit policy not rejecting allocations by default tends to be more useful than harmful. One example is forking processes: the memory space gets duplicated with copy-on-write, so if you didn't allow overcommit you could end up with processes that couldn't be forked due to that behavior. Not to mention that most programs don't use all the memory they have assigned, so without overcommit you'd have a lot of problems with RAM underusage. > So I find it extremely ironic for Linus to argue that practical programming languages are hard to use for environments that must handle OOM errors, when they are championing one of the major platforms that makes handling OOM useless by default. Kernel and user space programming are very, very different. A lot of developers use languages with garbage collection without issues, but that's not acceptable in kernel programming. The fact that the platform provides X feature does not mean it has to be developed including X feature. In this case it's very clear. Even if they used overcommit in kernel code, Linux does not always overcommit memory, so you'd still need to manage the OOM case in kernel properly.
deleted 6y ago
[deleted]

teknopaul 6y ago

You can't solve memory allocation problems by turning off overcommit and handling alloc errors in rust. You don't get Null, you get panic. That's a problem for the kernel, even if you argue it's not a problem elsewhere. Start killing drivers is nuts, oh look my screen went blank, and three HDs stopped working. Damn where is my swap.

volta83 6y ago
> Start killing drivers is nuts, I don't disagree, just saying that its as nuts as killing random user-space programs on OOM. "The web browser misbehaved, lets kill the pulseaudio daemon, and if that doesn't fix it, lets kill WiFi, and if that doesn't fix it, lets kill vim, and... " FFS if some app tried to allocate too much memory just let the app know, or kill that app, but don't start randomly killing user-space processes.
- simiones 6y ago
  While I agree that OOMKiller is pretty mad, it's also important to note that the shared nature of memory means that the app which will die when the system is out of memory is anyway random. Even without overcommit, you can get into the situation of "the web browser is occupying 99% of RAM to open YouTube, and now pulseaudio tried to allocate another 50 bytes that the system just doesn't have, so pulseaudio should handle this, and then the WiFi manager will be the next to need 100 bytes and be refused etc". Even worse, Linux by design stalls to a complete crawl when memory almost runs out, even with 0 swap and a fast disk, as it will first start swapping out code pages before giving up. Which means that every time a process is context switched, its code ends up first needing to be read from disk, the worst possible kind of cache thrashing imaginable.
  - Too 6y ago
    Somehow a LRU-algorithm makes the most sense, if only Linux and humans would have a common definition of what "recently used" means here. And if applications had safe-shutdown and restart on demand mechanisms. On a high level of abstraction this is how a lot of VMs deal with things, such as tabs in your browser or open apps on your smartphone.

nabla9 6y ago

You are talking about userspace. Linus is talking about kernel. You can change the Linux memory overcommit policy using sysctls if you need it.

teknopaul 6y ago
die on run out of memory is a bad policy for a userspace server. Seen some hairy production incidents with that, basically solution is piss off customers until they stop coming back. action on run out of memory should be finish up currently running work and hold off new stuff until you have resources, not die and start to reprocess the thundering herd. When you run out of memory last thing you want to do is empty your hot caches.
- bronson 6y ago
  You’re out of memory. Every possible cache has already been freed. Unless your app is faking it’s own caching, in which case it might be contributing to the problem.
  - angry_octet 6y ago
    It's quite possible that an app is holding onto data that can be flushed, at the expense of future database lookups, session negotiations, etc. Without a proper mechanism in the kernel it is hard to manage memory pressure.
- nabla9 6y ago
  If that would be important you would disable overcommit in Linux. If you didn't do that, it's probably because it's good tradeoff that saves resource.

pornel 6y ago

The "but the OOM killer!" argument has been a disaster for Rust, and keeps derailing all design discussions. • Linux is not the only OS in the world. • Even on Linux you have containers/cgroups that can impose hard limits. • Platforms without virtual memory are also an important target for Rust. • On 32-bit platforms you can run out of address space before you run out of RAM. • Regardless of what the OS does, the application may still want to impose its own internal limit (e.g. https://lib.rs/cap https://lib.rs/cap) to avoid being OOM-killed or swap death. People keep telling me how Linux never runs out of memory, while I'm currently firefighting a torrent of coredumps caused by Rust's self-own on OOM that actually happens. I generally love Rust, but its OOM handling is awful, and the "but the OOM killer" nonsense is to blame for stalling the absolutely critical fixes it urgently needs.

teknopaul 6y ago
word
varajelle 6y ago
The overcommit is not the only argument. The second argument is that complex programs that need to handle OOM properly will be full of complex untested error paths. How should one handle a OOM error If one can't allocate some temporary node during an algorithm and how to recover without leaving a corrupted state? That's not a question I want to answer for almost every function call. That's why the rust standard library data structures don't handle OOM. To make it easier to use for the most common case. And for the cases where the OOM handling is required, one can simply use data structures for it, which is what Linux will do (problem solved) Also compare that to the case of mutex poisoning which can return an error on each unlock, and which is now considered a mistake because in practice, nobody want to handle a poisoned mutex error.
- pornel 6y ago
  I argue it's a rare-enough problem: • Statistically, likelihood of failure is proportional to the allocation size. You'll most often fail to allocate a very large Vec, and then only need to allocate a few bytes for a string that says "error!". I'm using https://lib.rs/fallible_collections https://lib.rs/fallible_collections and just a few strategically placed try_reserve has brought down my coredumps by 99%. • Panic on OOM would unwind, and unwinding tends to free memory rather than allocate anything. The standard library should reserve/preallocate enough memory to be able to start unwinding. • Rust is good at avoiding temporary allocations and relies on the stack a lot. You don't have to use fancy error-handling libraries that collect backtrace on every error. Error handling with enums doesn't touch the heap at all. • Even if everything fails, and you get OOM during OOM handling, that's fair, and still better than abort every time. Note that Rust currently doesn't handle OOM by panicking. It unconditionally aborts the whole process. Libstd could switch to panics without API changes, and that would make OOM handling possible via catch_unwind.

sai_c 6y ago

So, basically you are saying "Rust is mostly used as an application programming language anyways, so let's not pester those users with the overhead of OOM handling"? Fair enough. But this would mean that Rust is (factually) a systems language in the same sense that Go was initially declared to be a systems language. If I look at the Rust community (I give it a try from time to time), I would totally agree with your point, that most users would be pestered by this overhead. I see mostly CLI tools, "web apps" and the kind. Furthermore, I just can't shake off the feeling that the embedded (bare metal, 16 or 32 bit architectures) or (in-house) kernel crowd will always be 2nd class citizens. I keep saying this, but the obvious fix is for the Linux kernel to use overcommit internally just like they expect user space to do. I'm not a kernel developer, but aren't you (maybe) asking for a bit too much? You ask the Linux kernel devs to change the kernel in a significant way, just so you can use Rust for writing drivers in a way you write user space applications.

dathinab 6y ago
No it's about "by default" not pestering users. Rust has all the tools needed to gracefully handle memory allocation failure. Yes, the tools are a bit limit wrt. usages in the standard library where you mainly can use them to handle "known to potentially fail big" allocations but not every single small allocation. But enter no_std and it's basically all your choice, which is also the default thing to use for embedded/bar-metal and I count the kernel as embedded/bar-metal. (It's default because the std library types are tuned for the most common use-cases including web-server and user-space system programming, in which you always can have panic on OOM, and error kernel as a much more ergonomic pattern then explicitly returning a result on every thing which potentially allocates). Through without question the discussion around it is a mess.
volta83 6y ago
> So, basically you are saying "Rust is mostly used as an application programming language anyways, so let's not pester those users with the overhead of OOM handling"? No. What I am saying is that Rust intended to be a systems programming language that could handle OOM properly everywhere, but that got a lot of opposition because some operating systems, mainly Linux, make it impossible for programs to handle OOM _at all_, so adding proper OOM support to Rust would mean that every Rust Linux programmer would be paying for a feature (in ergonomics, etc.) that they cannot use _at all_. That's the irony. Linux design has made it not worth it for new programming languages to be designed to handle OOM properly, because that's impossible to do in OSes like Linux. People have been complaining about this for user space programs for the last 20 years, and the Linux kernel stand was "that's a feature, not a bug". But now that they are confronted with it, you see Linus writing stuff like "that's not acceptable". That's a huge double standard IMO.

gjulianm 6y ago

> For example, on Linux with overcommit (the default), even if the system is out of memory, malloc won't return null. It returns a pointer that's not null, such that your if(ptr == nullptr) will act as if everything is "ok", Not always. The default overcommit policy will reject certain allocations that are too big. Also, you can change the overcommit policy if you need it. Also, overcommit policy not rejecting allocations by default tends to be more useful than harmful. One example is forking processes: the memory space gets duplicated with copy-on-write, so if you didn't allow overcommit you could end up with processes that couldn't be forked due to that behavior. Not to mention that most programs don't use all the memory they have assigned, so without overcommit you'd have a lot of problems with RAM underusage. > So I find it extremely ironic for Linus to argue that practical programming languages are hard to use for environments that must handle OOM errors, when they are championing one of the major platforms that makes handling OOM useless by default. Kernel and user space programming are very, very different. A lot of developers use languages with garbage collection without issues, but that's not acceptable in kernel programming. The fact that the platform provides X feature does not mean it has to be developed including X feature. In this case it's very clear. Even if they used overcommit in kernel code, Linux does not always overcommit memory, so you'd still need to manage the OOM case in kernel properly.

deleted 6y ago

[deleted]