4 ms·

> Drives me batty to see articles where malloc() is happily assumed to always succeed. ... as in Linux.

ycombobreaker 12y ago
Use `ulimit` from the shell, or invoke `setrlimit`, and you will see malloc fail.
mikeash 12y ago
Malloc will never fail on Linux due to running out of RAM. However, it will fail for other reasons, such as running out of address space. You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less. It's much harder in 64-bit processes, but since you can allocate vast chunks of memory without having them be backed by RAM, you can accomplish it there as well.
- linuxlizard 12y ago
  While address space is huge, RAM isn't infinite. Our embedded Linux runs out of memory quite often. Or if the swap space fills up on a server with a disk. I've spent many hours chasing corrupted memory due to someone not checking malloc() fail. I chased code that boiled down to assert( ptr=malloc() ); Guess what happens when NDEBUG is defined?
- jerf 12y ago
  "You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less." Ah, 21st Century problems.
  - linuxlizard 12y ago
    I know, right!? Kids these days. Sheesh. :-)
- antimagic 12y ago
  It's not so much that malloc can fail, but that you're likely to get whacked by OOM before you run out of address space on linux. To get malloc to return a NULL, you would have to be allocating more than 4Gb of memory on a 32 bit system, the system has to have no swap, and you have to not have used any of the memory between the last physically available memory, and the memory allocated by the chunk that takes you over the 4Gb limit. Anything else gets you SIGKILLed.
  - mikeash 12y ago
    Swap has nothing to do with it. Getting malloc to fail is easy: malloc(-1); With address space fragmentation, the requested size can be a lot smaller and still fail. With overcommit, you can allocate a ton of memory without using up any RAM.
linuxlizard 12y ago
Memory leaks in Firefox/Chrome might belie that notion. I've had Python throw MemoryError at me while I was doing some video processing with NumPy. 8GB RAM, 64GB swap. I wasn't careful with my refs and I confused Python's garbage collection (memory leak).

Use `ulimit` from the shell, or invoke `setrlimit`, and you will see malloc fail.

Malloc will never fail on Linux due to running out of RAM. However, it will fail for other reasons, such as running out of address space. You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less. It's much harder in 64-bit processes, but since you can allocate vast chunks of memory without having them be backed by RAM, you can accomplish it there as well.

linuxlizard 12y ago
While address space is huge, RAM isn't infinite. Our embedded Linux runs out of memory quite often. Or if the swap space fills up on a server with a disk. I've spent many hours chasing corrupted memory due to someone not checking malloc() fail. I chased code that boiled down to assert( ptr=malloc() ); Guess what happens when NDEBUG is defined?
jerf 12y ago
"You can easily run out of address space in 32-bit processes, since you only have 4GB of address space, or less." Ah, 21st Century problems.
- linuxlizard 12y ago
  I know, right!? Kids these days. Sheesh. :-)
antimagic 12y ago
It's not so much that malloc can fail, but that you're likely to get whacked by OOM before you run out of address space on linux. To get malloc to return a NULL, you would have to be allocating more than 4Gb of memory on a 32 bit system, the system has to have no swap, and you have to not have used any of the memory between the last physically available memory, and the memory allocated by the chunk that takes you over the 4Gb limit. Anything else gets you SIGKILLed.
- mikeash 12y ago
  Swap has nothing to do with it. Getting malloc to fail is easy: malloc(-1); With address space fragmentation, the requested size can be a lot smaller and still fail. With overcommit, you can allocate a ton of memory without using up any RAM.

linuxlizard 12y ago

Memory leaks in Firefox/Chrome might belie that notion. I've had Python throw MemoryError at me while I was doing some video processing with NumPy. 8GB RAM, 64GB swap. I wasn't careful with my refs and I confused Python's garbage collection (memory leak).