3 ms·

What sort of C programmer wouldn't test for malloc() failing. Drives me batty to see articles where malloc() is happily assumed to always succeed.

ExpiredLink 12y ago
> Drives me batty to see articles where malloc() is happily assumed to always succeed. ... as in Linux.
- ycombobreaker 12y ago
  Use `ulimit` from the shell, or invoke `setrlimit`, and you will see malloc fail.
- mikeash 12y ago
  Malloc will never fail on Linux due to running out of RAM. However, it will fail for other reasons, such as running out of address space. You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less. It's much harder in 64-bit processes, but since you can allocate vast chunks of memory without having them be backed by RAM, you can accomplish it there as well.
  - linuxlizard 12y ago
    While address space is huge, RAM isn't infinite. Our embedded Linux runs out of memory quite often. Or if the swap space fills up on a server with a disk. I've spent many hours chasing corrupted memory due to someone not checking malloc() fail. I chased code that boiled down to assert( ptr=malloc() ); Guess what happens when NDEBUG is defined?
  - jerf 12y ago
    "You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less." Ah, 21st Century problems.
    - linuxlizard 12y ago
      I know, right!? Kids these days. Sheesh. :-)
  - antimagic 12y ago
    It's not so much that malloc can fail, but that you're likely to get whacked by OOM before you run out of address space on linux. To get malloc to return a NULL, you would have to be allocating more than 4Gb of memory on a 32 bit system, the system has to have no swap, and you have to not have used any of the memory between the last physically available memory, and the memory allocated by the chunk that takes you over the 4Gb limit. Anything else gets you SIGKILLed.
    - mikeash 12y ago
      Swap has nothing to do with it. Getting malloc to fail is easy: malloc(-1); With address space fragmentation, the requested size can be a lot smaller and still fail. With overcommit, you can allocate a ton of memory without using up any RAM.
- linuxlizard 12y ago
  Memory leaks in Firefox/Chrome might belie that notion. I've had Python throw MemoryError at me while I was doing some video processing with NumPy. 8GB RAM, 64GB swap. I wasn't careful with my refs and I confused Python's garbage collection (memory leak).
kabdib 12y ago
Rule #1 of systems programming: "Never check for an error that you don't know how to handle." This is, of course, a joke, but it has a kernel of truth. I've worked on production code where, if allocation fails, the system fails, too, and restarts. The allocator never returns. There's usually some kind of helper system watching for the failure, which can take a dump or at least a stack trace. The theory is that any attempt at recovery is going to be a disaster; not well tested, and probably doomed in the face of memory pressure anyway. On 64 bit systems the memory ceiling is essentially infinite, but reaching the point where we start to swap is horrible and a pyrrhic victory at best, so instead let's do a controlled crash and log enough information so we can analyze and fix the actual problem (which is a misunderstanding of load and resources, or maybe a bug or an attack where we're trying to allocate twenty gazillion bytes). That's one philosophy. Another (which was prevalent on the Mac in the 80s, in user applications) is to have a reserve; you test the crap out of the code and give it enough memory in some kind of emergency mode to succeed at getting back to a normal state, or at least quitting in an orderly fashion (ditch dynamically loaded code, fonts, whatever) so that the user's work isn't totally lost. There's a lot to be said for both designs, and of course these don't cover the case when you're writing library code that can be used by anyone, where reliably returning failure to the caller is important. Here you don't have a choice other than to check and be robust. So three, yes, three philosophies.
- linuxlizard 12y ago
  I worked on some safety critical systems in C. The philosophy was treat the code like a little person running around. The person (the code) always wanted to know where the fire escape was. The fire escape always needed to be very easy to use and very reliable. Every function had little fire escape handler to clean up on error. Higher level functions had to decide how to put out the fire (speaking metaphorically here but we really could have started fires) or to raise the alarm to a higher level. At the very top level, the only thing we could do was save a log then reset (least worst option). Too many resets too fast and we'd take ourselves offline and scream for a human (again, least worst option).
fsloth 12y ago
If malloc fails for trying to allocate a few miserable bytes I would claim the game is over. Why would he need a fallback strategy, anyway? The program is tiny and really, really doesn't do anything important. Matters of taste I suppose, but I've always felt that for tutorial and proof of concept code it's better to skip unnecessry details. Not every line of code lives in a production environment...

ExpiredLink 12y ago

> Drives me batty to see articles where malloc() is happily assumed to always succeed. ... as in Linux.

ycombobreaker 12y ago
Use `ulimit` from the shell, or invoke `setrlimit`, and you will see malloc fail.
mikeash 12y ago
Malloc will never fail on Linux due to running out of RAM. However, it will fail for other reasons, such as running out of address space. You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less. It's much harder in 64-bit processes, but since you can allocate vast chunks of memory without having them be backed by RAM, you can accomplish it there as well.
- linuxlizard 12y ago
  While address space is huge, RAM isn't infinite. Our embedded Linux runs out of memory quite often. Or if the swap space fills up on a server with a disk. I've spent many hours chasing corrupted memory due to someone not checking malloc() fail. I chased code that boiled down to assert( ptr=malloc() ); Guess what happens when NDEBUG is defined?
- jerf 12y ago
  "You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less." Ah, 21st Century problems.
  - linuxlizard 12y ago
    I know, right!? Kids these days. Sheesh. :-)
- antimagic 12y ago
  It's not so much that malloc can fail, but that you're likely to get whacked by OOM before you run out of address space on linux. To get malloc to return a NULL, you would have to be allocating more than 4Gb of memory on a 32 bit system, the system has to have no swap, and you have to not have used any of the memory between the last physically available memory, and the memory allocated by the chunk that takes you over the 4Gb limit. Anything else gets you SIGKILLed.
  - mikeash 12y ago
    Swap has nothing to do with it. Getting malloc to fail is easy: malloc(-1); With address space fragmentation, the requested size can be a lot smaller and still fail. With overcommit, you can allocate a ton of memory without using up any RAM.
linuxlizard 12y ago
Memory leaks in Firefox/Chrome might belie that notion. I've had Python throw MemoryError at me while I was doing some video processing with NumPy. 8GB RAM, 64GB swap. I wasn't careful with my refs and I confused Python's garbage collection (memory leak).

kabdib 12y ago

Rule #1 of systems programming: "Never check for an error that you don't know how to handle." This is, of course, a joke, but it has a kernel of truth. I've worked on production code where, if allocation fails, the system fails, too, and restarts. The allocator never returns. There's usually some kind of helper system watching for the failure, which can take a dump or at least a stack trace. The theory is that any attempt at recovery is going to be a disaster; not well tested, and probably doomed in the face of memory pressure anyway. On 64 bit systems the memory ceiling is essentially infinite, but reaching the point where we start to swap is horrible and a pyrrhic victory at best, so instead let's do a controlled crash and log enough information so we can analyze and fix the actual problem (which is a misunderstanding of load and resources, or maybe a bug or an attack where we're trying to allocate twenty gazillion bytes). That's one philosophy. Another (which was prevalent on the Mac in the 80s, in user applications) is to have a reserve; you test the crap out of the code and give it enough memory in some kind of emergency mode to succeed at getting back to a normal state, or at least quitting in an orderly fashion (ditch dynamically loaded code, fonts, whatever) so that the user's work isn't totally lost. There's a lot to be said for both designs, and of course these don't cover the case when you're writing library code that can be used by anyone, where reliably returning failure to the caller is important. Here you don't have a choice other than to check and be robust. So three, yes, three philosophies.

linuxlizard 12y ago
I worked on some safety critical systems in C. The philosophy was treat the code like a little person running around. The person (the code) always wanted to know where the fire escape was. The fire escape always needed to be very easy to use and very reliable. Every function had little fire escape handler to clean up on error. Higher level functions had to decide how to put out the fire (speaking metaphorically here but we really could have started fires) or to raise the alarm to a higher level. At the very top level, the only thing we could do was save a log then reset (least worst option). Too many resets too fast and we'd take ourselves offline and scream for a human (again, least worst option).

fsloth 12y ago

If malloc fails for trying to allocate a few miserable bytes I would claim the game is over. Why would he need a fallback strategy, anyway? The program is tiny and really, really doesn't do anything important. Matters of taste I suppose, but I've always felt that for tutorial and proof of concept code it's better to skip unnecessry details. Not every line of code lives in a production environment...