7 ms·

"Discipline" is indeed the right word. I was taught C at Epitech. A single segfault, no matter insidious, was a valid reason to render a whole project NULL. We

"Discipline" is indeed the right word. I was taught C at Epitech. A single segfault, no matter insidious, was a valid reason to render a whole project NULL. We often had evaluations ran with LD_LIBRATY_PATH=malloc_that_fails.so or just piping /dev/urandom to stdin... Needless to say, calling exit() when a call to malloc() failed wasn't an acceptable recover routine.

Crito 13y ago
> Needless to say, calling exit() when a call to malloc() failed wasn't an acceptable recover routine. What do you do when malloc fails? A bit of graceful shutdown and logging seems like it would be in order, but otherwise how do you keep rolling if mallocs start failing? It seems to me like that would indicate something has gone unusually wrong and full recovery is futile.
- ethomson 13y ago
  I don't get the sense that there was any attempt to recover from errors - it sounds more like they were enforcing that error checking occurred, by replacing `malloc` with one that just returned `NULL` always. It sounds like the goal was to make sure that one didn't assume `malloc` would always succeed and just use the memory. Indeed, recovery is basically futile in this case and your program is going to shut down pretty quickly either way. Maybe you'll get the chance to tell the user that you ran out of memory before you die, which seems polite.
  - cpeterso 13y ago
    In systems that overcommit memory (like Linux), malloc() can return non-NULL and then crash when you read or write that address because the system doesn't have enough real memory to back that virtual address.
    - deathanatos 13y ago
      Even in Linux which set to overcommit, malloc() can still return NULL if you exhaust the virtual address space of your process, though I expect it's much less likely now on 64-bit platforms.
  - alextingle 13y ago
    What if you application is the fly-by-wire for an airliner? Can you imagine that there might be better options than just calling abort(3)?
    - clarry 13y ago
      Yes, a better option is to make sure this error cannot happen, by making sure the program has enough memory to begin with. Fly-by-wire shouldn't need unbounded memory allocations at runtime. There are some applications where you can try to recover by freeing something that isn't critical, or by waiting and trying again. Or you can gracefully fail whatever computation is going on right now, without aborting the entire program. But these are last resort things and will not always save you. If your fly-by-wire ever depends on such a last resort, it's broken by design :-)
    - sitkack 13y ago
      Embedded systems like this do not use dynamic memory allocation.
      - rmrfrmrf 13y ago
        Ugh that's why I'll never buy Boeing; non-upgradable memory in 2013? Please!
      - Crito 13y ago
        Related: The JPL C guidelines forbid dynamic memory allocation: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf (HN discussion: https://news.ycombinator.com/item?id=4339999 https://news.ycombinator.com/item?id=4339999) They also demand that code be littered with sanity checking assertions.
        sitkack 13y ago
        Thanks for posting this. Tangent, I was thinking about Toyota's software process failure and how they _invented_ industrial level mistake proofing yet did not apply it their engine throttle code. C is obviously the wrong language, but from a software perspective they should have at least tested the engine controller from an adversarial standpoint (salt water on the board, stuck sensors). That is the crappy thing about Harvard architecture cpu (separate instruction and data memory), you can have while loops that NEVER crash controlling a machine that continues to wreck havoc, sometimes you want a hard reset and a fast recovery. http://en.wikipedia.org/wiki/Crash-only_software http://en.wikipedia.org/wiki/Crash-only_software
  - joshguthrie 13y ago
    Like eliteraspberrie said, the proper way to recover from an error is to unroll your stack back to your main function and return 1 there. Error checking was enforced for EVERY syscall, be it malloc() or open(). Checking for errors was indeed required but not enough: proper and graceful shut down was required too.
- eliteraspberrie 13y ago
  The most common way to indicate that an error has occurred in a C function is to return non-zero. If this is done consistently, and return values are always checked, an error condition will eventually make its way up to main, where you can fail gracefully. For example: int a(void) { ... if (oops) { return 1; } return 0; } int b(void) { if (a(...) != 0) { return 1; } return 0; } int main(void) { if (b(...) != 0) { exit(EXIT_FAILURE); } return EXIT_SUCCESS; } (This means that void functions should be avoided.)
  - clarry 13y ago
    What's the point of propagating all errors way up to main if you're only going to exit anyway? I think we know how to indicate errors in C functions. What to do about specific errors, in this case allocation failures, is a more interesting question.
    - joshguthrie 13y ago
      To make sure you free() all your previous allocations on the way down. You can choose not too, it's "kinda" the same (can't remember the exact difference) but it's dirty and people with OCD just won't accept it. (Disclosure: I got OCD too, this is not meant to make C development hostile to people with OCD.)
      - scott_s 13y ago
        If your program is going to exit anyway, there's no need to call free() on allocated memory. The operating system will reclaim all of the memory it granted to your process when it exits. Remember that malloc lives in your process, not the kernel. When you ask for memory from malloc, it is actually doling out memory that you already own - the operating system granted it to you, when malloc requested it. And malloc requested it because you asked for more memory than it was currently managing. If your intention is to continue running, then of course you want to call free() on your memory. And this certainly makes sense to do as you exit functions. But if you're, say, calling exit() in the middle of your program, for whatever reason, you don't need to worry about memory. Other resources may be a problem, though. The the operating system will reclaim things it controlled, and granted to your process - memory, sockets, file descriptors and such. But you need to be careful about resources not controlled by the operating system in such a manner.
- joshguthrie 13y ago
  Go ahead. Call exit(). On your HTTP/IRC/anything server. Just because you couldn't allocate memory for one more client. Now your service is down and the CTO is looking for blood =) Yes, it's far-fetched and like some said further down the comments, you "can't" run out of memory in Linux, but straight killing a service is never good.
  - astrange 13y ago
    If your server's running Limux, it's going to kill your process with no questions asked if you run out of memory. You're better off practicing crash-only error recovery and having robust clients that can handle a reconnect. HTTP is stateless already, so crash and restart all you want!
    - yew 13y ago
      The OOM killer is more likely to kill some other process and trash your server. Thankfully that sort of behavior has been vastly reduced since the thing was introduced, but disabling overcommit for high-reliability applications is still a reasonable course of action.
      - dingaling 13y ago
        The OOM killer might eventually kill something, after it thrashes the system for a few hours. I had a server last week in which swap hadn't been configured. A compilation job took all memory and the OOM started thrashing. Thankfully there's always one SSH session open but I couldn't kill anything, sync or shutdown; fork failed with insufficient memory. Left if thrashing overnight and had to power-kill it next day.
  - Crito 13y ago
    I'd say letting that server bounce and having the watchdog/load balancer work to keep you at capacity is the best option there. You are going to need that infrastructure anyway and if you can't malloc enough for one more client, who is to say that the existing stuff is going to be able to continue either? You should count on any particular instance bouncing when that happens, and design your system to survive that. You should also invest some effort to figure out why your system can get into that state. Consider if any particular instance should be attempting to accept more jobs than it has the ability to handle. I shouldn't be able to trigger an out of memory situation on your IRC server just by connecting as many irc clients as I can.
- vidarh 13y ago
  I grew up using the Amiga, when having memory allocation fail was routine (a standard Amiga 500 for example, came with 512KB RAM, and was rarely expanded to more than 1MB, so you would run out of memory). What you do when malloc() fails depends entirely on your application: If a desktop application on the Amiga would shutdown just because a memory allocation failed, nobody would use it. The expection was you'd gracefully clean up, and fail whatever operation needed the memory, and if possible inform the user to let him/her free up memory before trying again. This expectations in "modern" OS's that malloc never fails unless the world is falling really annoys me - it for example leads to systems where we use too much swap to the point where systems often slow down or become hopelessly unresponsive in cases where the proper response would have been to inform the user - the user experience is horrendous: Swap is/was a kludge to handle high memory prices; having the option is great, but most of the time when I have systems that dip into swap, it indicates a problem I'd want to be informed about. But on modern systems, most software handles it so badly that turning swap off is often not even a viable choice. Of course there are plenty of situations where the above isn't the proper response, e.g. where you can't ask the user. But even for many servers, the proper response would not be fall over and die if you can reasonably dial back your resource usage and fail in more graceful ways. E.g. an app server does a better job if it at least provides the option to dynamically scale back the number of connections it handles rather than failing to provide service at all - degrading service or slowing down is often vastly better than having a service fail entirely.
  - MichaelGG 13y ago
    Isn't fork the real offender, which requires Linux to overcommit by default? Disabling swap shouldn't affect that, right? Just makes your problem happen later, in a somewhat non-deterministic way. Without fork, what reason do you not disable swap? I can only think of an anonymous mmap where you want to use the OS VM as a cache system. But that's solved easily enough by providing a backing file, isn't it?
    - daeken 13y ago
      > Isn't fork the real offender, which requires Linux to overcommit by default? Maybe I'm missing something, but how does fork require overcommitment? When you fork, you end up with COW pages, which share underlying memory. They don't guarantee that physical memory would be available if every page were touched and required a copy; they just share underlying physical memory. Strictly speaking, very little allocation has to happen for a process fork to occur.
housel 13y ago
I suspect you mean LD_PRELOAD=whatever.so
- joshguthrie 13y ago
  Thanks, haven't used it in two years and am a bit rusty :)

Crito 13y ago

> Needless to say, calling exit() when a call to malloc() failed wasn't an acceptable recover routine. What do you do when malloc fails? A bit of graceful shutdown and logging seems like it would be in order, but otherwise how do you keep rolling if mallocs start failing? It seems to me like that would indicate something has gone unusually wrong and full recovery is futile.

ethomson 13y ago
I don't get the sense that there was any attempt to recover from errors - it sounds more like they were enforcing that error checking occurred, by replacing `malloc` with one that just returned `NULL` always. It sounds like the goal was to make sure that one didn't assume `malloc` would always succeed and just use the memory. Indeed, recovery is basically futile in this case and your program is going to shut down pretty quickly either way. Maybe you'll get the chance to tell the user that you ran out of memory before you die, which seems polite.
- cpeterso 13y ago
  In systems that overcommit memory (like Linux), malloc() can return non-NULL and then crash when you read or write that address because the system doesn't have enough real memory to back that virtual address.
  - deathanatos 13y ago
    Even in Linux which set to overcommit, malloc() can still return NULL if you exhaust the virtual address space of your process, though I expect it's much less likely now on 64-bit platforms.
- alextingle 13y ago
  What if you application is the fly-by-wire for an airliner? Can you imagine that there might be better options than just calling abort(3)?
  - clarry 13y ago
    Yes, a better option is to make sure this error cannot happen, by making sure the program has enough memory to begin with. Fly-by-wire shouldn't need unbounded memory allocations at runtime. There are some applications where you can try to recover by freeing something that isn't critical, or by waiting and trying again. Or you can gracefully fail whatever computation is going on right now, without aborting the entire program. But these are last resort things and will not always save you. If your fly-by-wire ever depends on such a last resort, it's broken by design :-)
  - sitkack 13y ago
    Embedded systems like this do not use dynamic memory allocation.
    - rmrfrmrf 13y ago
      Ugh that's why I'll never buy Boeing; non-upgradable memory in 2013? Please!
    - Crito 13y ago
      Related: The JPL C guidelines forbid dynamic memory allocation: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf (HN discussion: https://news.ycombinator.com/item?id=4339999 https://news.ycombinator.com/item?id=4339999) They also demand that code be littered with sanity checking assertions.
      - sitkack 13y ago
        Thanks for posting this. Tangent, I was thinking about Toyota's software process failure and how they _invented_ industrial level mistake proofing yet did not apply it their engine throttle code. C is obviously the wrong language, but from a software perspective they should have at least tested the engine controller from an adversarial standpoint (salt water on the board, stuck sensors). That is the crappy thing about Harvard architecture cpu (separate instruction and data memory), you can have while loops that NEVER crash controlling a machine that continues to wreck havoc, sometimes you want a hard reset and a fast recovery. http://en.wikipedia.org/wiki/Crash-only_software http://en.wikipedia.org/wiki/Crash-only_software
- joshguthrie 13y ago
  Like eliteraspberrie said, the proper way to recover from an error is to unroll your stack back to your main function and return 1 there. Error checking was enforced for EVERY syscall, be it malloc() or open(). Checking for errors was indeed required but not enough: proper and graceful shut down was required too.
eliteraspberrie 13y ago
The most common way to indicate that an error has occurred in a C function is to return non-zero. If this is done consistently, and return values are always checked, an error condition will eventually make its way up to main, where you can fail gracefully. For example: int a(void) { ... if (oops) { return 1; } return 0; } int b(void) { if (a(...) != 0) { return 1; } return 0; } int main(void) { if (b(...) != 0) { exit(EXIT_FAILURE); } return EXIT_SUCCESS; } (This means that void functions should be avoided.)
- clarry 13y ago
  What's the point of propagating all errors way up to main if you're only going to exit anyway? I think we know how to indicate errors in C functions. What to do about specific errors, in this case allocation failures, is a more interesting question.
  - joshguthrie 13y ago
    To make sure you free() all your previous allocations on the way down. You can choose not too, it's "kinda" the same (can't remember the exact difference) but it's dirty and people with OCD just won't accept it. (Disclosure: I got OCD too, this is not meant to make C development hostile to people with OCD.)
    - scott_s 13y ago
      If your program is going to exit anyway, there's no need to call free() on allocated memory. The operating system will reclaim all of the memory it granted to your process when it exits. Remember that malloc lives in your process, not the kernel. When you ask for memory from malloc, it is actually doling out memory that you already own - the operating system granted it to you, when malloc requested it. And malloc requested it because you asked for more memory than it was currently managing. If your intention is to continue running, then of course you want to call free() on your memory. And this certainly makes sense to do as you exit functions. But if you're, say, calling exit() in the middle of your program, for whatever reason, you don't need to worry about memory. Other resources may be a problem, though. The the operating system will reclaim things it controlled, and granted to your process - memory, sockets, file descriptors and such. But you need to be careful about resources not controlled by the operating system in such a manner.
joshguthrie 13y ago
Go ahead. Call exit(). On your HTTP/IRC/anything server. Just because you couldn't allocate memory for one more client. Now your service is down and the CTO is looking for blood =) Yes, it's far-fetched and like some said further down the comments, you "can't" run out of memory in Linux, but straight killing a service is never good.
- astrange 13y ago
  If your server's running Limux, it's going to kill your process with no questions asked if you run out of memory. You're better off practicing crash-only error recovery and having robust clients that can handle a reconnect. HTTP is stateless already, so crash and restart all you want!
  - yew 13y ago
    The OOM killer is more likely to kill some other process and trash your server. Thankfully that sort of behavior has been vastly reduced since the thing was introduced, but disabling overcommit for high-reliability applications is still a reasonable course of action.
    - dingaling 13y ago
      The OOM killer might eventually kill something, after it thrashes the system for a few hours. I had a server last week in which swap hadn't been configured. A compilation job took all memory and the OOM started thrashing. Thankfully there's always one SSH session open but I couldn't kill anything, sync or shutdown; fork failed with insufficient memory. Left if thrashing overnight and had to power-kill it next day.
- Crito 13y ago
  I'd say letting that server bounce and having the watchdog/load balancer work to keep you at capacity is the best option there. You are going to need that infrastructure anyway and if you can't malloc enough for one more client, who is to say that the existing stuff is going to be able to continue either? You should count on any particular instance bouncing when that happens, and design your system to survive that. You should also invest some effort to figure out why your system can get into that state. Consider if any particular instance should be attempting to accept more jobs than it has the ability to handle. I shouldn't be able to trigger an out of memory situation on your IRC server just by connecting as many irc clients as I can.
vidarh 13y ago
I grew up using the Amiga, when having memory allocation fail was routine (a standard Amiga 500 for example, came with 512KB RAM, and was rarely expanded to more than 1MB, so you would run out of memory). What you do when malloc() fails depends entirely on your application: If a desktop application on the Amiga would shutdown just because a memory allocation failed, nobody would use it. The expection was you'd gracefully clean up, and fail whatever operation needed the memory, and if possible inform the user to let him/her free up memory before trying again. This expectations in "modern" OS's that malloc never fails unless the world is falling really annoys me - it for example leads to systems where we use too much swap to the point where systems often slow down or become hopelessly unresponsive in cases where the proper response would have been to inform the user - the user experience is horrendous: Swap is/was a kludge to handle high memory prices; having the option is great, but most of the time when I have systems that dip into swap, it indicates a problem I'd want to be informed about. But on modern systems, most software handles it so badly that turning swap off is often not even a viable choice. Of course there are plenty of situations where the above isn't the proper response, e.g. where you can't ask the user. But even for many servers, the proper response would not be fall over and die if you can reasonably dial back your resource usage and fail in more graceful ways. E.g. an app server does a better job if it at least provides the option to dynamically scale back the number of connections it handles rather than failing to provide service at all - degrading service or slowing down is often vastly better than having a service fail entirely.
- MichaelGG 13y ago
  Isn't fork the real offender, which requires Linux to overcommit by default? Disabling swap shouldn't affect that, right? Just makes your problem happen later, in a somewhat non-deterministic way. Without fork, what reason do you not disable swap? I can only think of an anonymous mmap where you want to use the OS VM as a cache system. But that's solved easily enough by providing a backing file, isn't it?
  - daeken 13y ago
    > Isn't fork the real offender, which requires Linux to overcommit by default? Maybe I'm missing something, but how does fork require overcommitment? When you fork, you end up with COW pages, which share underlying memory. They don't guarantee that physical memory would be available if every page were touched and required a copy; they just share underlying physical memory. Strictly speaking, very little allocation has to happen for a process fork to occur.

housel 13y ago

I suspect you mean LD_PRELOAD=whatever.so

joshguthrie 13y ago
Thanks, haven't used it in two years and am a bit rusty :)