5 ms·

This is downvoted, but I'd rather see it refuted. Is this false? Is SQLite as written tolerant of malloc() failure?

steveklabnik 11y ago
The Rust standard library does assume that allocation succeeds. The language itself knows nothing about the heap, and so you can write allocators that do whatever you wish. Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure. I am less knowledgeable about OSX and Windows. I cannot speak to SQlite in this regard.
- aidenn0 11y ago
  Even with overcommit on, malloc can return NULL on linux.
  - steveklabnik 11y ago
    liballoc does check for NULL IIRC.
- ktRolster 11y ago
  Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure Clarifying note: this is configurable.
- nickpsecurity 11y ago
  Holy crap, I'd never have imagined that... A design that assumes allocation always succeeds flies in the face of decades of safety/security-critical coding wisdom. I strongly recommend the team revisit and change that somehow to account for failures, NULL's, whatever. Actually, same anywhere a failure-prone, esp hardware, resource is acquired. C apps can handle this issue so Rust should as well if it's to replace them. EDIT: Thanks to replies for clarification that it's just one allocator, aborts, and others are available. Still feel weird about it but that's better.
  - kinghajj 11y ago
    Note that Rust, the language, is perfectly capable of handling memory allocation failures, it's just the standard library that makes the assumption. Embedded environments wouldn't use the standard library for numerous reasons anyway, and the "core" library uses no allocation at all. That said, IIRC handling of failure in the standard library will happen eventually, I believe there's just no consensus yet on the best way to do it.
  - tatterdemalion 11y ago
    The standard allocator will abort on oom error. For applications which need to be tolerant to oom errors, you need to use a different allocator. Most programs can't tolerate oom errors, though, and it would be absolutely unreasonable for every function in the standard library that might perform an allocation to return an error that has to be handled by every programmer all the time. EDIT: C's solution is to make it very easy to ignore oom errors, so most programmers just don't handle oom errors. Rust's solution is much better.
    - jstimpfle 11y ago
      To be fair, not a bug in the language itself. Regarding libc, it's also very easy to abort on error. Just write a function xmalloc().
    - quotemstr 11y ago
      Every function in the C++ standard library reliably communicates allocation failure to the application without relying on aborting the whole program. If Rust can do it, C++ can do it too. Rust got itself into this trap by eschewing exceptions.
      - tatterdemalion 11y ago
        Maybe avoid using judgmental language like calling a standard allocator that aborts on oom a "trap." The Rust team made conscious design choices in full awareness of the trade-offs. Moreover, Rust actually does have thread unwinding, and even the ability to catch an unwinding thread, so it is not true that the Rust standard library could not have unwound on oom. Rust's standard library just isn't designed for writing applications that need to survive oom errors. That's fine, its not designed for a number of other applications which Rust the language is well-suited for either (operating systems, for example). Its designed for the majority use case, because life is full of trade offs.
deleted 11y ago
[deleted]

The Rust standard library does assume that allocation succeeds. The language itself knows nothing about the heap, and so you can write allocators that do whatever you wish. Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure. I am less knowledgeable about OSX and Windows. I cannot speak to SQlite in this regard.

aidenn0 11y ago
Even with overcommit on, malloc can return NULL on linux.
- steveklabnik 11y ago
  liballoc does check for NULL IIRC.
ktRolster 11y ago
Side note: on most Linux distros, overcommit is on, and so malloc will basically always succeed; the OOM killer will kill your program before you'd get a failure Clarifying note: this is configurable.
nickpsecurity 11y ago
Holy crap, I'd never have imagined that... A design that assumes allocation always succeeds flies in the face of decades of safety/security-critical coding wisdom. I strongly recommend the team revisit and change that somehow to account for failures, NULL's, whatever. Actually, same anywhere a failure-prone, esp hardware, resource is acquired. C apps can handle this issue so Rust should as well if it's to replace them. EDIT: Thanks to replies for clarification that it's just one allocator, aborts, and others are available. Still feel weird about it but that's better.
- kinghajj 11y ago
  Note that Rust, the language, is perfectly capable of handling memory allocation failures, it's just the standard library that makes the assumption. Embedded environments wouldn't use the standard library for numerous reasons anyway, and the "core" library uses no allocation at all. That said, IIRC handling of failure in the standard library will happen eventually, I believe there's just no consensus yet on the best way to do it.
- tatterdemalion 11y ago
  The standard allocator will abort on oom error. For applications which need to be tolerant to oom errors, you need to use a different allocator. Most programs can't tolerate oom errors, though, and it would be absolutely unreasonable for every function in the standard library that might perform an allocation to return an error that has to be handled by every programmer all the time. EDIT: C's solution is to make it very easy to ignore oom errors, so most programmers just don't handle oom errors. Rust's solution is much better.
  - jstimpfle 11y ago
    To be fair, not a bug in the language itself. Regarding libc, it's also very easy to abort on error. Just write a function xmalloc().
  - quotemstr 11y ago
    Every function in the C++ standard library reliably communicates allocation failure to the application without relying on aborting the whole program. If Rust can do it, C++ can do it too. Rust got itself into this trap by eschewing exceptions.
    - tatterdemalion 11y ago
      Maybe avoid using judgmental language like calling a standard allocator that aborts on oom a "trap." The Rust team made conscious design choices in full awareness of the trade-offs. Moreover, Rust actually does have thread unwinding, and even the ability to catch an unwinding thread, so it is not true that the Rust standard library could not have unwound on oom. Rust's standard library just isn't designed for writing applications that need to survive oom errors. That's fine, its not designed for a number of other applications which Rust the language is well-suited for either (operating systems, for example). Its designed for the majority use case, because life is full of trade offs.

deleted 11y ago

[deleted]