4 ms·

Handling OOM gracefully - i.e. doing anything other than immediately crashing and/or invoking undefined behaviour - is absolutely not the default in C. It's po

by gpm 10mo ago

Handling OOM gracefully - i.e. doing anything other than immediately crashing and/or invoking undefined behaviour - is absolutely not the default in C. It's possible. But very very few projects do.

lelanthran 10mo ago
> Handling OOM gracefully - i.e. doing anything other than immediately crashing and/or invoking undefined behaviour - is absolutely not the default in C. What are you talking about? Every allocation must be checked at the point of allocation, which is "the default" If you write non-idiomatically, then sure, in other languages you can jump through a couple of hoops and check every allocation, but that's not the default. The default in C is to return an error when allocation fails. The default in C++, Rust, etc is to throw an exception. The idiomatic way in C++, etc is to not handle that exception.
- gpm 10mo ago
  > Every allocation must be checked at the point of allocation, which is "the default" C doesn't force you to check the allocation at all. The default behavior is to simply invoke undefined behavior the first time you use the returned allocation if it failed. In practice I've found most people write their own wrappers around malloc that at least crash - for example: https://docs.gtk.org/glib/memory.html https://docs.gtk.org/glib/memory.html PS. The current default in rust to print something and then abort the program, not panic (i.e. not throw an exception). Though the standard library reserves the right to change that to a panic in the future.
  - lelanthran 10mo ago
    > C doesn't force you to check the allocation at all. No one ever claimed it did; I said, and still do, that the in C, at any rate, the default is to check the returned value from memory allocations. And, that is true. The default in other language is not to recover.
    - gpm 10mo ago
      > > C doesn't force you to check the allocation at all. > No one ever claimed it did; You specifically said > Every allocation must be checked at the point of allocation ... > the default is to check the returned value from memory allocations. Default has a meaning, and it's what happens if you don't explicitly choose to do something else. In libc - this is to invoke undefined behavior if the user uses the allocation. In glib - the library that underpins half the linux desktop - this is to crash. This is an approach I've seen elsewhere as well to the point where I'm comfortable calling it "default" in the sense that people change their default behavior to it. Nowhere that I've ever seen, in C, is it to make the user handle the error. I assume there are projects with santizers that do do that, I haven't worked on them, and they certainly don't make up the majority.
      - lelanthran 10mo ago
        > Default has a meaning, and it's what happens if you don't explicitly choose to do something else. It also has the meaning of doing the common thing: https://www.merriam-webster.com/dictionary/default https://www.merriam-webster.com/dictionary/default > : a selection made usually automatically or without active consideration See that "without active consideration" there? The default usage of malloc includes, whether you want to acknowledge it or not, checking the returned value. C doesn't have anything done automatically, so I am wondering why you would choose to think that by "default" one would mean that something automatically gets done.
        gpm 10mo ago
        I'm not saying "automatic", I'm including "sanitizer retursn an error" as default - that's not what happens in C (or at least any C project I've worked on). You have to actively remember and choose to check the error code. Of course things do happen automatically all the time in C, like bumping the stack pointer (another case of unhandled OOM) and decrementing it after the fact. And pushing return addresses - and returning at the end of functions. And so on. "Ubiquitous" is a different word than default, checking the return code of malloc isn't even that. As an example - I've been having some issues with pipewire recently (unrelated) and happen to know it uses an unwrapped malloc. And it doesn't reliably check the return code. For example: https://github.com/PipeWire/pipewire/blob/6ed964546586e809f72d4668268b09f47987b080/pipewire-jack/src/metadata.c#L314 https://github.com/PipeWire/pipewire/blob/6ed964546586e809f7... And again, this isn't cherry picked, this is just "the last popular open source C code base I've looked at". This is the common case in C. Either you wrap malloc to crash, or you just accept undefined behavior if malloc fails. It is the rare project that doesn't do one of those two.
- joosters 10mo ago
  From the parent comment: because of OS-level overcommit, which is nearly always a good thing It doesn't matter about the language you are writing in, because your OS can tell you that the allocation succeeded, but when you come to use it, only then do you find out that the memory isn't there.
  - senko 10mo ago
    Of course it matters, because you (the system admin) can tell your OS not to do that. Which is only helpful if your app knows how to handle the case. Most don't, so overcommit, in general, makes sense.
    - deleted 10mo ago
      [deleted]
    - gpm 10mo ago
      You can't really on linux. There's no way to do sparse allocations then because when you turn off overcommit MAP_NORESERVE still reserves memory... It's a place where windows legitimately is better than linux.
      - lelanthran 10mo ago
        > You can't really on linux. There's no way to do sparse allocations then because when you turn off overcommit MAP_NORESERVE still reserves memory... Sure, but ... what does that have to do with this thread? Using `mmap` is not the same as using `malloc` and friends. If you turn off overcommit, malloc will return NULL on failure to allocate. If you specifically request mmap to ignore overcommit, and it does, why are you surprised?
        gpm 10mo ago
        > If you specifically request mmap to ignore overcommit, and it does, why are you surprised? You misunderstand, you specifically request mmap to ignore overcommit, and it doesn't, not does. What it has to do with this thread is it makes turning off overcommit on linux an exceptionally unpalatable option because it makes a lot of correct software incorrect in an unfixable manner.
koakuma-chan 10mo ago
I know one C++ library that caches data but never evicts. Instead, the library author expects you to restart your app every 24 hours.
- lelanthran 10mo ago
  > I know one C++ library that caches data but never evicts. Instead, the library author expects you to restart your app every 24 hours. It may not be as simple as "that's our policy". I worked at one place (embedded C++ code, 2018) that simply reset the device every 24h because they never managed to track down all the leaks. Finding memory leaks in C++ is a non-trivial and time-consuming task. It gets easier if your project doesn't use exceptions, but it's still very difficult.
  - williamoforange 10mo ago
    Use Valgrind? Or are we talking projects that have become far too big for their own good, cause leaks aren't hard at all to find with the right tools and a bit of profiling... now crossing thread boundaries and weird dynamic programming tricks maybe, but thats a very different case and not really refecting on C++ itself, would likely trip up a GC lang as well.
    - lelanthran 10mo ago
      > Use Valgrind? Was not available for that specific device, but even with Valgrind and similar tools, you are still going to run into weird destructor issues with inheritance. There are many possible combinations of virtual, non-virtual, base-class, derived-class, constructors and destructors; some of them will indeed cause a memory leak, and are allowed to by the standard.
      - paulf38 10mo ago
        > even with Valgrind and similar tools, you are still going to run into weird destructor issues with inheritance. I love these folklore comments. Post an example.

by gpm 10mo ago

Handling OOM gracefully - i.e. doing anything other than immediately crashing and/or invoking undefined behaviour - is absolutely not the default in C. It's possible. But very very few projects do.

lelanthran 10mo ago

> Handling OOM gracefully - i.e. doing anything other than immediately crashing and/or invoking undefined behaviour - is absolutely not the default in C. What are you talking about? Every allocation must be checked at the point of allocation, which is "the default" If you write non-idiomatically, then sure, in other languages you can jump through a couple of hoops and check every allocation, but that's not the default. The default in C is to return an error when allocation fails. The default in C++, Rust, etc is to throw an exception. The idiomatic way in C++, etc is to not handle that exception.

gpm 10mo ago
> Every allocation must be checked at the point of allocation, which is "the default" C doesn't force you to check the allocation at all. The default behavior is to simply invoke undefined behavior the first time you use the returned allocation if it failed. In practice I've found most people write their own wrappers around malloc that at least crash - for example: https://docs.gtk.org/glib/memory.html https://docs.gtk.org/glib/memory.html PS. The current default in rust to print something and then abort the program, not panic (i.e. not throw an exception). Though the standard library reserves the right to change that to a panic in the future.
- lelanthran 10mo ago
  > C doesn't force you to check the allocation at all. No one ever claimed it did; I said, and still do, that the in C, at any rate, the default is to check the returned value from memory allocations. And, that is true. The default in other language is not to recover.
  - gpm 10mo ago
    > > C doesn't force you to check the allocation at all. > No one ever claimed it did; You specifically said > Every allocation must be checked at the point of allocation ... > the default is to check the returned value from memory allocations. Default has a meaning, and it's what happens if you don't explicitly choose to do something else. In libc - this is to invoke undefined behavior if the user uses the allocation. In glib - the library that underpins half the linux desktop - this is to crash. This is an approach I've seen elsewhere as well to the point where I'm comfortable calling it "default" in the sense that people change their default behavior to it. Nowhere that I've ever seen, in C, is it to make the user handle the error. I assume there are projects with santizers that do do that, I haven't worked on them, and they certainly don't make up the majority.
    - lelanthran 10mo ago
      > Default has a meaning, and it's what happens if you don't explicitly choose to do something else. It also has the meaning of doing the common thing: https://www.merriam-webster.com/dictionary/default https://www.merriam-webster.com/dictionary/default > : a selection made usually automatically or without active consideration See that "without active consideration" there? The default usage of malloc includes, whether you want to acknowledge it or not, checking the returned value. C doesn't have anything done automatically, so I am wondering why you would choose to think that by "default" one would mean that something automatically gets done.
      - gpm 10mo ago
        I'm not saying "automatic", I'm including "sanitizer retursn an error" as default - that's not what happens in C (or at least any C project I've worked on). You have to actively remember and choose to check the error code. Of course things do happen automatically all the time in C, like bumping the stack pointer (another case of unhandled OOM) and decrementing it after the fact. And pushing return addresses - and returning at the end of functions. And so on. "Ubiquitous" is a different word than default, checking the return code of malloc isn't even that. As an example - I've been having some issues with pipewire recently (unrelated) and happen to know it uses an unwrapped malloc. And it doesn't reliably check the return code. For example: https://github.com/PipeWire/pipewire/blob/6ed964546586e809f72d4668268b09f47987b080/pipewire-jack/src/metadata.c#L314 https://github.com/PipeWire/pipewire/blob/6ed964546586e809f7... And again, this isn't cherry picked, this is just "the last popular open source C code base I've looked at". This is the common case in C. Either you wrap malloc to crash, or you just accept undefined behavior if malloc fails. It is the rare project that doesn't do one of those two.
joosters 10mo ago
From the parent comment: because of OS-level overcommit, which is nearly always a good thing It doesn't matter about the language you are writing in, because your OS can tell you that the allocation succeeded, but when you come to use it, only then do you find out that the memory isn't there.
- senko 10mo ago
  Of course it matters, because you (the system admin) can tell your OS not to do that. Which is only helpful if your app knows how to handle the case. Most don't, so overcommit, in general, makes sense.
  - deleted 10mo ago
    [deleted]
  - gpm 10mo ago
    You can't really on linux. There's no way to do sparse allocations then because when you turn off overcommit MAP_NORESERVE still reserves memory... It's a place where windows legitimately is better than linux.
    - lelanthran 10mo ago
      > You can't really on linux. There's no way to do sparse allocations then because when you turn off overcommit MAP_NORESERVE still reserves memory... Sure, but ... what does that have to do with this thread? Using `mmap` is not the same as using `malloc` and friends. If you turn off overcommit, malloc will return NULL on failure to allocate. If you specifically request mmap to ignore overcommit, and it does, why are you surprised?
      - gpm 10mo ago
        > If you specifically request mmap to ignore overcommit, and it does, why are you surprised? You misunderstand, you specifically request mmap to ignore overcommit, and it doesn't, not does. What it has to do with this thread is it makes turning off overcommit on linux an exceptionally unpalatable option because it makes a lot of correct software incorrect in an unfixable manner.

koakuma-chan 10mo ago

I know one C++ library that caches data but never evicts. Instead, the library author expects you to restart your app every 24 hours.

lelanthran 10mo ago
> I know one C++ library that caches data but never evicts. Instead, the library author expects you to restart your app every 24 hours. It may not be as simple as "that's our policy". I worked at one place (embedded C++ code, 2018) that simply reset the device every 24h because they never managed to track down all the leaks. Finding memory leaks in C++ is a non-trivial and time-consuming task. It gets easier if your project doesn't use exceptions, but it's still very difficult.
- williamoforange 10mo ago
  Use Valgrind? Or are we talking projects that have become far too big for their own good, cause leaks aren't hard at all to find with the right tools and a bit of profiling... now crossing thread boundaries and weird dynamic programming tricks maybe, but thats a very different case and not really refecting on C++ itself, would likely trip up a GC lang as well.
  - lelanthran 10mo ago
    > Use Valgrind? Was not available for that specific device, but even with Valgrind and similar tools, you are still going to run into weird destructor issues with inheritance. There are many possible combinations of virtual, non-virtual, base-class, derived-class, constructors and destructors; some of them will indeed cause a memory leak, and are allowed to by the standard.
    - paulf38 10mo ago
      > even with Valgrind and similar tools, you are still going to run into weird destructor issues with inheritance. I love these folklore comments. Post an example.