4 ms·

It's definitely very important for some domains, but handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming. Modern op

by dbaupp 11y ago

It's definitely very important for some domains, but handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming. Modern operating systems will over-allocate/over-commit memory via virtual memory (so malloc usually succeeds), and kill programs when the physical memory limit is reached, and/or just grind to a halt by swapping. For instance, I can happily "allocate" 100 terabytes of memory on my laptop (16 GB of RAM, 16 of swap): use std::{thread, time}; fn main() { let _v: Vec<Vec<u8>> = (0..100_000) .map(|_| Vec::with_capacity(1 << 30)) // 1GB each .collect(); thread::sleep(time::Duration::from_secs(10)); } Of course, if I actually tried to use much of it, bad things would happen, but the point is the allocations themselves never failed. Bare-metal programming, like a kernel, will typically use the lower-level `core` part of the standard library, rather than the whole thing (i.e. `std`) because the kernel will be literally providing the interfaces required for most of std-minus-core. (I think it would be really great if the standard library could be less harsh when an OOM is actually detected, and I suspect it can be tuned in future, but it's not a show-stopper for most tasks.)

quotemstr 11y ago
> handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming I strongly disagree. Handling memory allocation failure is trivial: throw an exception. It's also trivial to reserve enough memory to throw an exception. I honestly don't understand people who claim that recovering from OOM is futile because you need to almost always allocate memory to recover: in fact, you almost never do. > Modern operating systems will over-allocate/over-commit memory Windows is a very modern operating system and does not overcommit. (To its credit: overcommit is a terrible idea.) Even on overcommit systems, you can exhaust your address space and see allocation failures. No, 64 bits is not actually infinity. No, the whole world is not 64 bit. (Not every program even benefits from being 64 bit --- that's why the x32 ABI exists.) > point is the allocations themselves never failed. The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. It's very difficult to explain to people who work exclusively on overcommit systems that "memory" isn't a single resource, but a set of related ones, each of which can be exhausted. > I think it would be really great if the standard library could be less harsh when an OOM is actually detected The only reasonable thing to do on OOM failure is throw an exception, like Java and C++ do. Error codes are too cumbersome even if you dress them up and call them "monads" --- a lot of code allocates, so if you used error codes to indicate OOM, you'd have to manually propagate exceptions from almost everywhere) and just panicking or aborting is too brittle, since you can, in fact, successfully recover from allocation failure, persistent folk myths aside. Now Rust has both Result _and_ panic/catch_panic, which I think is both sad and hilarious, as it's going to lead to decades of confusion over which error reporting mechanism to use in a particular situation. (The Go designers made exactly the same mistake, as did the designers of the old java.io.File interface.) It's a shame for such a young language to have that kind of baggage already. If Rust's designers hadn't been so stridently anti-exception in the first place, we'd have ended up with a much better language.
- kibwen 11y ago
  > Now Rust has both Result _and_ panic/catch_panic The forthcoming facilities for "catching" panics are deliberately hobbled so as to prevent them from being suitable for use as a general-purpose error-handling mechanism. What they're for is fault isolation, most specifically for preventing code from unwinding across a FFI boundary (which would result in undefined behavior).
  - quotemstr 11y ago
    catch_panic doesn't look particularly hobbled to me. Besides, I can insert an "FFI" boundary between any two parts of the program I want, and nothing can stop me. But it's moot: I'm not keen on using a language designed by people who are opposed in principle to exceptions, the cleanest form of error handling around.
    - kibwen 11y ago
      You can read the comments on the original RFC for stabilizing catch support and see that the majority of them revolve around various ways of deliberately handicapping the API: https://github.com/rust-lang/rfcs/pull/1236 https://github.com/rust-lang/rfcs/pull/1236 It even inspired a followup RFC for the purpose of codifying these restrictions in a brand-new fundamental trait: https://github.com/rust-lang/rfcs/pull/1323 https://github.com/rust-lang/rfcs/pull/1323
      - quotemstr 11y ago
        Yes, but it's still progress toward the inevitable end result of traditional C++-style exceptions. That's the inevitable end state because the traditional exception model is simply the best one.
        kibwen 11y ago
        > the inevitable It's not inevitable in the slightest. Rust made a conscious effort to avoid exceptions for good reasons, and there's no reason to suspect that Rust will suddenly pivot on its heel in the opposite direction. Despite your attempts to paint this as a slippery slope, the unwinding-halting mechanisms are there to provide a safety net for people writing Rust libraries that expose C ABIs so that library code can fail gracefully without taking down the parent process. > the traditional exception model is simply the best one Citation needed. :P The Rust community has long since embraced Result-based error handling; I don't hear calls for adding exceptions nearly as often as you seem to suspect. As to "inevitability", you'll be sad to learn that Rust plans on adding a mechanism that will enable applications to make panics impossible to catch, by disabling unwinding entirely and translating all panics to outright aborts (some argue that this should even be the default behavior of Rust code, but that won't happen any time soon). And considering that Mozilla disables exceptions entirely in Firefox, I see no reason they would pressure the Rust developers to add exceptions to Rust. You seem to be taking Rust's approach to error handling far too personally. :P It is not Rust's position that exceptions are a blight that must be scoured from the face of the Earth. It is merely Rust's position that there are easier ways to achieve concise and composable error handling for its intended domain, given Rust's other features and safety mechanisms.

by dbaupp 11y ago

quotemstr 11y ago

> handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming I strongly disagree. Handling memory allocation failure is trivial: throw an exception. It's also trivial to reserve enough memory to throw an exception. I honestly don't understand people who claim that recovering from OOM is futile because you need to almost always allocate memory to recover: in fact, you almost never do. > Modern operating systems will over-allocate/over-commit memory Windows is a very modern operating system and does not overcommit. (To its credit: overcommit is a terrible idea.) Even on overcommit systems, you can exhaust your address space and see allocation failures. No, 64 bits is not actually infinity. No, the whole world is not 64 bit. (Not every program even benefits from being 64 bit --- that's why the x32 ABI exists.) > point is the allocations themselves never failed. The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. It's very difficult to explain to people who work exclusively on overcommit systems that "memory" isn't a single resource, but a set of related ones, each of which can be exhausted. > I think it would be really great if the standard library could be less harsh when an OOM is actually detected The only reasonable thing to do on OOM failure is throw an exception, like Java and C++ do. Error codes are too cumbersome even if you dress them up and call them "monads" --- a lot of code allocates, so if you used error codes to indicate OOM, you'd have to manually propagate exceptions from almost everywhere) and just panicking or aborting is too brittle, since you can, in fact, successfully recover from allocation failure, persistent folk myths aside. Now Rust has both Result _and_ panic/catch_panic, which I think is both sad and hilarious, as it's going to lead to decades of confusion over which error reporting mechanism to use in a particular situation. (The Go designers made exactly the same mistake, as did the designers of the old java.io.File interface.) It's a shame for such a young language to have that kind of baggage already. If Rust's designers hadn't been so stridently anti-exception in the first place, we'd have ended up with a much better language.

kibwen 11y ago
> Now Rust has both Result _and_ panic/catch_panic The forthcoming facilities for "catching" panics are deliberately hobbled so as to prevent them from being suitable for use as a general-purpose error-handling mechanism. What they're for is fault isolation, most specifically for preventing code from unwinding across a FFI boundary (which would result in undefined behavior).
- quotemstr 11y ago
  catch_panic doesn't look particularly hobbled to me. Besides, I can insert an "FFI" boundary between any two parts of the program I want, and nothing can stop me. But it's moot: I'm not keen on using a language designed by people who are opposed in principle to exceptions, the cleanest form of error handling around.
  - kibwen 11y ago
    You can read the comments on the original RFC for stabilizing catch support and see that the majority of them revolve around various ways of deliberately handicapping the API: https://github.com/rust-lang/rfcs/pull/1236 https://github.com/rust-lang/rfcs/pull/1236 It even inspired a followup RFC for the purpose of codifying these restrictions in a brand-new fundamental trait: https://github.com/rust-lang/rfcs/pull/1323 https://github.com/rust-lang/rfcs/pull/1323
    - quotemstr 11y ago
      Yes, but it's still progress toward the inevitable end result of traditional C++-style exceptions. That's the inevitable end state because the traditional exception model is simply the best one.
      - kibwen 11y ago
        > the inevitable It's not inevitable in the slightest. Rust made a conscious effort to avoid exceptions for good reasons, and there's no reason to suspect that Rust will suddenly pivot on its heel in the opposite direction. Despite your attempts to paint this as a slippery slope, the unwinding-halting mechanisms are there to provide a safety net for people writing Rust libraries that expose C ABIs so that library code can fail gracefully without taking down the parent process. > the traditional exception model is simply the best one Citation needed. :P The Rust community has long since embraced Result-based error handling; I don't hear calls for adding exceptions nearly as often as you seem to suspect. As to "inevitability", you'll be sad to learn that Rust plans on adding a mechanism that will enable applications to make panics impossible to catch, by disabling unwinding entirely and translating all panics to outright aborts (some argue that this should even be the default behavior of Rust code, but that won't happen any time soon). And considering that Mozilla disables exceptions entirely in Firefox, I see no reason they would pressure the Rust developers to add exceptions to Rust. You seem to be taking Rust's approach to error handling far too personally. :P It is not Rust's position that exceptions are a blight that must be scoured from the face of the Earth. It is merely Rust's position that there are easier ways to achieve concise and composable error handling for its intended domain, given Rust's other features and safety mechanisms.