7 ms·

Yes: now we all forget how to correctly manage memory. Rust: where resources are unlimited and aborting when you run out of them is okay

pcwalton 10y ago
"Rust: where resources are unlimited and aborting when you run out of them is okay" I'd change "Rust" to "safe programming" here, but OK. There have been lots of fun security vulnerabilities coming from programs trying to handle OOM gracefully. e.g. SpiderMonkey: https://bugzilla.mozilla.org/show_bug.cgi?id=982957 https://bugzilla.mozilla.org/show_bug.cgi?id=982957, https://bugzilla.mozilla.org/show_bug.cgi?id=730415 https://bugzilla.mozilla.org/show_bug.cgi?id=730415
Manishearth 10y ago
Rust has provided ways to not abort on oom for a while now.
- wahern 10y ago
  If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one or the other when panic'ing. Also, programs can be built in such a way that all panics become aborts. That kind of sucks but it's probably one of the compromises needed to assuage those Rust developers who don't believe it's practical to handle OOM conditions.
  - steveklabnik 10y ago
    To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others. The default allocator crate always does an abort, both before and after this change.
    - deleted 10y ago
      [deleted]
    - deleted 10y ago
      [deleted]
    - wahern 10y ago
      When building an app, does changing the default allocator effect all imported libraries? And is that allocator used for boxing, when boxing requires dynamic allocation?
      - steveklabnik 10y ago
        Currently, "the allocator" is global in Rust. That is, the standard library uses liballoc to do all of its heap allocation, and it's liballoc that aborts on OOM. You can of course write code that does anything, including use some other mechanism than liballoc. There's motion on several fronts here: The first, and currently unstable, is swapping out the implementation of liballoc for a different one. (we ship jemalloc and a pass-through to the system allocator with Rust) The second, and currently in the "working on an RFC" phase, is per-object allocators. Both of these are unstable because we're not 100% sure of the interface we want to stabilize yet, it's still a work in progress. > when boxing requires dynamic allocation? Boxing always implies dynamic allocation.
vertex-four 10y ago
The vast majority of programs that you run on a day to day basis will abort when they run out of memory. On a default Linux system, there's not even any way to prevent that - you need to faff with the config to make it actually return errors when allocating memory.
- wahern 10y ago
  That's all irrelevant for a language which is supposed to be a systems programming language, however that term is defined.
  - vertex-four 10y ago
    I don't think it is - less, bash, gcc, systemd, most interpreters, and Firefox will all pretty much just crash if malloc() fails - they're varying levels of things you'd want a "systems language" for. Unless you're trying to define systems language to mean one used for kernel or bare-metal embedded development (both cases where even in C, you skip the standard library and write your own memory allocation functions etc), there's really very few cases where you (a) are allowed to allocate memory and (b) need to fail gracefully somehow if you can't.
    - wahern 10y ago
      I'm not saying that aborting on OOM never makes sense. I'm saying there are many situations where you shouldn't abort on OOM, and those situations overlap considerably with the situations in which a systems language is ideal[1]. So a systems language that doesn't allow for robust and fine-grained OOM handing isn't much of a systems language. Example: Basically any highly concurrent network daemon that multiplexes many clients on the same thread. In that case, you want much more control over where to put your recovery point. Even if the process doesn't abort, if the recovery point is beyond the scope of the kernel thread (i.e. a controller thread, which was the recommended solution before catch_unwind), that can be really inconvenient, and also requires a lot of unnecessary passing of mutable state between threads, which is usually something you try to avoid. [1] Lua has robust support for OOM recovery, which is noteworthy because there can be situations where you both want to handle OOM but where a scripting language is more preferable. Example: An image manipulation program with scriptable filters, where you don't want an operation that can't complete to take down your process or thread.
deleted 10y ago
[deleted]

pcwalton 10y ago

"Rust: where resources are unlimited and aborting when you run out of them is okay" I'd change "Rust" to "safe programming" here, but OK. There have been lots of fun security vulnerabilities coming from programs trying to handle OOM gracefully. e.g. SpiderMonkey: https://bugzilla.mozilla.org/show_bug.cgi?id=982957 https://bugzilla.mozilla.org/show_bug.cgi?id=982957, https://bugzilla.mozilla.org/show_bug.cgi?id=730415 https://bugzilla.mozilla.org/show_bug.cgi?id=730415

Manishearth 10y ago

Rust has provided ways to not abort on oom for a while now.

wahern 10y ago
If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one or the other when panic'ing. Also, programs can be built in such a way that all panics become aborts. That kind of sucks but it's probably one of the compromises needed to assuage those Rust developers who don't believe it's practical to handle OOM conditions.
- steveklabnik 10y ago
  To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others. The default allocator crate always does an abort, both before and after this change.
  - deleted 10y ago
    [deleted]
  - deleted 10y ago
    [deleted]
  - wahern 10y ago
    When building an app, does changing the default allocator effect all imported libraries? And is that allocator used for boxing, when boxing requires dynamic allocation?
    - steveklabnik 10y ago
      Currently, "the allocator" is global in Rust. That is, the standard library uses liballoc to do all of its heap allocation, and it's liballoc that aborts on OOM. You can of course write code that does anything, including use some other mechanism than liballoc. There's motion on several fronts here: The first, and currently unstable, is swapping out the implementation of liballoc for a different one. (we ship jemalloc and a pass-through to the system allocator with Rust) The second, and currently in the "working on an RFC" phase, is per-object allocators. Both of these are unstable because we're not 100% sure of the interface we want to stabilize yet, it's still a work in progress. > when boxing requires dynamic allocation? Boxing always implies dynamic allocation.

vertex-four 10y ago

The vast majority of programs that you run on a day to day basis will abort when they run out of memory. On a default Linux system, there's not even any way to prevent that - you need to faff with the config to make it actually return errors when allocating memory.

wahern 10y ago
That's all irrelevant for a language which is supposed to be a systems programming language, however that term is defined.
- vertex-four 10y ago
  I don't think it is - less, bash, gcc, systemd, most interpreters, and Firefox will all pretty much just crash if malloc() fails - they're varying levels of things you'd want a "systems language" for. Unless you're trying to define systems language to mean one used for kernel or bare-metal embedded development (both cases where even in C, you skip the standard library and write your own memory allocation functions etc), there's really very few cases where you (a) are allowed to allocate memory and (b) need to fail gracefully somehow if you can't.
  - wahern 10y ago
    I'm not saying that aborting on OOM never makes sense. I'm saying there are many situations where you shouldn't abort on OOM, and those situations overlap considerably with the situations in which a systems language is ideal[1]. So a systems language that doesn't allow for robust and fine-grained OOM handing isn't much of a systems language. Example: Basically any highly concurrent network daemon that multiplexes many clients on the same thread. In that case, you want much more control over where to put your recovery point. Even if the process doesn't abort, if the recovery point is beyond the scope of the kernel thread (i.e. a controller thread, which was the recommended solution before catch_unwind), that can be really inconvenient, and also requires a lot of unnecessary passing of mutable state between threads, which is usually something you try to avoid. [1] Lua has robust support for OOM recovery, which is noteworthy because there can be situations where you both want to handle OOM but where a scripting language is more preferable. Example: An image manipulation program with scriptable filters, where you don't want an operation that can't complete to take down your process or thread.

deleted 10y ago

[deleted]