3 ms·

> handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming I strongly disagree. Handling memory allocation failure is

> handling OOM "properly" is a really vague/impossible concept for most non-bare-metal programming I strongly disagree. Handling memory allocation failure is trivial: throw an exception. It's also trivial to reserve enough memory to throw an exception. I honestly don't understand people who claim that recovering from OOM is futile because you need to almost always allocate memory to recover: in fact, you almost never do. > Modern operating systems will over-allocate/over-commit memory Windows is a very modern operating system and does not overcommit. (To its credit: overcommit is a terrible idea.) Even on overcommit systems, you can exhaust your address space and see allocation failures. No, 64 bits is not actually infinity. No, the whole world is not 64 bit. (Not every program even benefits from being 64 bit --- that's why the x32 ABI exists.) > point is the allocations themselves never failed. The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. It's very difficult to explain to people who work exclusively on overcommit systems that "memory" isn't a single resource, but a set of related ones, each of which can be exhausted. > I think it would be really great if the standard library could be less harsh when an OOM is actually detected The only reasonable thing to do on OOM failure is throw an exception, like Java and C++ do. Error codes are too cumbersome even if you dress them up and call them "monads" --- a lot of code allocates, so if you used error codes to indicate OOM, you'd have to manually propagate exceptions from almost everywhere) and just panicking or aborting is too brittle, since you can, in fact, successfully recover from allocation failure, persistent folk myths aside. Now Rust has both Result _and_ panic/catch_panic, which I think is both sad and hilarious, as it's going to lead to decades of confusion over which error reporting mechanism to use in a particular situation. (The Go designers made exactly the same mistake, as did the designers of the old java.io.File interface.) It's a shame for such a young language to have that kind of baggage already. If Rust's designers hadn't been so stridently anti-exception in the first place, we'd have ended up with a much better language.

kibwen 11y ago
> Now Rust has both Result _and_ panic/catch_panic The forthcoming facilities for "catching" panics are deliberately hobbled so as to prevent them from being suitable for use as a general-purpose error-handling mechanism. What they're for is fault isolation, most specifically for preventing code from unwinding across a FFI boundary (which would result in undefined behavior).
- quotemstr 11y ago
  catch_panic doesn't look particularly hobbled to me. Besides, I can insert an "FFI" boundary between any two parts of the program I want, and nothing can stop me. But it's moot: I'm not keen on using a language designed by people who are opposed in principle to exceptions, the cleanest form of error handling around.
  - kibwen 11y ago
    You can read the comments on the original RFC for stabilizing catch support and see that the majority of them revolve around various ways of deliberately handicapping the API: https://github.com/rust-lang/rfcs/pull/1236 https://github.com/rust-lang/rfcs/pull/1236 It even inspired a followup RFC for the purpose of codifying these restrictions in a brand-new fundamental trait: https://github.com/rust-lang/rfcs/pull/1323 https://github.com/rust-lang/rfcs/pull/1323
    - quotemstr 11y ago
      Yes, but it's still progress toward the inevitable end result of traditional C++-style exceptions. That's the inevitable end state because the traditional exception model is simply the best one.
      - kibwen 11y ago
        > the inevitable It's not inevitable in the slightest. Rust made a conscious effort to avoid exceptions for good reasons, and there's no reason to suspect that Rust will suddenly pivot on its heel in the opposite direction. Despite your attempts to paint this as a slippery slope, the unwinding-halting mechanisms are there to provide a safety net for people writing Rust libraries that expose C ABIs so that library code can fail gracefully without taking down the parent process. > the traditional exception model is simply the best one Citation needed. :P The Rust community has long since embraced Result-based error handling; I don't hear calls for adding exceptions nearly as often as you seem to suspect. As to "inevitability", you'll be sad to learn that Rust plans on adding a mechanism that will enable applications to make panics impossible to catch, by disabling unwinding entirely and translating all panics to outright aborts (some argue that this should even be the default behavior of Rust code, but that won't happen any time soon). And considering that Mozilla disables exceptions entirely in Firefox, I see no reason they would pressure the Rust developers to add exceptions to Rust. You seem to be taking Rust's approach to error handling far too personally. :P It is not Rust's position that exceptions are a blight that must be scoured from the face of the Earth. It is merely Rust's position that there are easier ways to achieve concise and composable error handling for its intended domain, given Rust's other features and safety mechanisms.
dbaupp 11y ago
> Handling memory allocation failure is trivial The problem I'm talking about is allocation failure and OOM are not the same: handling out-of-memory reliably is a lot more than just checking for NULL in `malloc`, as I demonstrated. There's no guarantee that allocations will fail when they "should" and so you'll only learn that you're beyond reasonable limits when you touch enough pages that the operating system starts killing programs (possibly yours, possibly others), or when the user cannot actually use their computer in any reasonable manner. > Windows is a very modern operating system and does not overcommit Yeah definitely, it's why I was sure to mention the problems with swapping (sure, the memory is allocated and usable by the program, but in slow motion; not so great if you actually want to use the computer). > The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. Yes, running out of virtual memory is one way for allocations to fail, but, as you say, there's a variety of different systems at play here. Only failure in one of those systems (requesting virtual memory/address space) is handled by checking for `malloc` returning NULL, and so focusing on just that is missing the forest for the trees. Presumably the overall goal is to handle memory exhaustion gracefully, not to check the return value of NULL (which is just one part of the former).
- quotemstr 11y ago
  That some operating systems lie about resource commitments is no reason not to provide reasonable facilities for reacting to allocation failures. By doing so, you make it harder to write well-behaved programs for systems that don't overcommit --- besides, no system overcommits address space, so malloc can always fail. It's true that even with reasonable allocation failure reporting on a non-overcommit system, you can use too much memory and make life unpleasant for users without actually seeing allocations fail. Bounding resource use is important regardless. That's an issue of good behavior, however. Even absent good behavior, programs should at least be robust. One of the reasons overcommit bothers me so much is that infects programming communities with the idea that resource consumption doesn't matter, since if you use too much, you're just going to get a SIGKILL anyway.
  - 11y ago

kibwen 11y ago

> Now Rust has both Result _and_ panic/catch_panic The forthcoming facilities for "catching" panics are deliberately hobbled so as to prevent them from being suitable for use as a general-purpose error-handling mechanism. What they're for is fault isolation, most specifically for preventing code from unwinding across a FFI boundary (which would result in undefined behavior).

quotemstr 11y ago
catch_panic doesn't look particularly hobbled to me. Besides, I can insert an "FFI" boundary between any two parts of the program I want, and nothing can stop me. But it's moot: I'm not keen on using a language designed by people who are opposed in principle to exceptions, the cleanest form of error handling around.
- kibwen 11y ago
  You can read the comments on the original RFC for stabilizing catch support and see that the majority of them revolve around various ways of deliberately handicapping the API: https://github.com/rust-lang/rfcs/pull/1236 https://github.com/rust-lang/rfcs/pull/1236 It even inspired a followup RFC for the purpose of codifying these restrictions in a brand-new fundamental trait: https://github.com/rust-lang/rfcs/pull/1323 https://github.com/rust-lang/rfcs/pull/1323
  - quotemstr 11y ago
    Yes, but it's still progress toward the inevitable end result of traditional C++-style exceptions. That's the inevitable end state because the traditional exception model is simply the best one.
    - kibwen 11y ago
      > the inevitable It's not inevitable in the slightest. Rust made a conscious effort to avoid exceptions for good reasons, and there's no reason to suspect that Rust will suddenly pivot on its heel in the opposite direction. Despite your attempts to paint this as a slippery slope, the unwinding-halting mechanisms are there to provide a safety net for people writing Rust libraries that expose C ABIs so that library code can fail gracefully without taking down the parent process. > the traditional exception model is simply the best one Citation needed. :P The Rust community has long since embraced Result-based error handling; I don't hear calls for adding exceptions nearly as often as you seem to suspect. As to "inevitability", you'll be sad to learn that Rust plans on adding a mechanism that will enable applications to make panics impossible to catch, by disabling unwinding entirely and translating all panics to outright aborts (some argue that this should even be the default behavior of Rust code, but that won't happen any time soon). And considering that Mozilla disables exceptions entirely in Firefox, I see no reason they would pressure the Rust developers to add exceptions to Rust. You seem to be taking Rust's approach to error handling far too personally. :P It is not Rust's position that exceptions are a blight that must be scoured from the face of the Earth. It is merely Rust's position that there are easier ways to achieve concise and composable error handling for its intended domain, given Rust's other features and safety mechanisms.

dbaupp 11y ago

> Handling memory allocation failure is trivial The problem I'm talking about is allocation failure and OOM are not the same: handling out-of-memory reliably is a lot more than just checking for NULL in `malloc`, as I demonstrated. There's no guarantee that allocations will fail when they "should" and so you'll only learn that you're beyond reasonable limits when you touch enough pages that the operating system starts killing programs (possibly yours, possibly others), or when the user cannot actually use their computer in any reasonable manner. > Windows is a very modern operating system and does not overcommit Yeah definitely, it's why I was sure to mention the problems with swapping (sure, the memory is allocated and usable by the program, but in slow motion; not so great if you actually want to use the computer). > The resource you're allocating is address space. When you run out of this resource, your allocations will fail. On overcommit systems, you don't allocate commit charge until you access the memory. Yes, running out of virtual memory is one way for allocations to fail, but, as you say, there's a variety of different systems at play here. Only failure in one of those systems (requesting virtual memory/address space) is handled by checking for `malloc` returning NULL, and so focusing on just that is missing the forest for the trees. Presumably the overall goal is to handle memory exhaustion gracefully, not to check the return value of NULL (which is just one part of the former).

quotemstr 11y ago
That some operating systems lie about resource commitments is no reason not to provide reasonable facilities for reacting to allocation failures. By doing so, you make it harder to write well-behaved programs for systems that don't overcommit --- besides, no system overcommits address space, so malloc can always fail. It's true that even with reasonable allocation failure reporting on a non-overcommit system, you can use too much memory and make life unpleasant for users without actually seeing allocations fail. Bounding resource use is important regardless. That's an issue of good behavior, however. Even absent good behavior, programs should at least be robust. One of the reasons overcommit bothers me so much is that infects programming communities with the idea that resource consumption doesn't matter, since if you use too much, you're just going to get a SIGKILL anyway.
- 11y ago