13 ms·
Malloc Never Fails (2012)
- vbezhenar 7y agomalloc fails with ulimit
- dullgiulio 7y agoVery relevant, because containers are often under what basically is ulimit. Not that you can do much when malloc fails...
- IcePic 7y ago"graceful degradation" is one of the things worth trying.
- chrisseaton 7y ago> Not that you can do much when malloc fails... Tell the user they can’t do that operation? Use on-disk storage instead? Re-use memory you already have (such as evicting a cache and taking the memory it already had allocated)? Abandon what you were doing if it was only an optimisation and wasn’t essential (such as allocating memory as part of a speculative execution)? Run a garbage collector and try again? Lots of options available in some situations.
- Crinus 7y agoIn practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some programs and try again" dialog but even those were hit and miss, depending on the situation. Once 32bit hardware-based virtual memory entered the picture, every single application would just assume you have endless RAM - memory checks are reserved for "common cases" like trying to create a 100000x100000 image in a (32bit) image editor. The reason for all that is simple: out of memory situations are stupidly and increasingly rare and in most cases where they can happen there isn't much you can do (e.g. what would you do if you run out of memory while making the fourth button in a toolbar?) and really in the 99% of the cases there might only be five people in the entire universe that will encounter such a case (two of them will tweet about it though and amass a lot of "lol, those garbage developers" retweets) so littering your codebase to keep those five people (and their Twitter followers) happy is not worth the effort. I mean, are you really going to put a "run out of memory" check after every toolbar button allocation? And what are you going to do if that fails? What can you do? AFAIK some modern languages nowadays even assume memory allocations wont fail (and if they do they just terminate).
- chrisseaton 7y ago> what would you do if you run out of memory while making the fourth button in a toolbar? Rollback what I’d created so far, evict caches, ask malloc to trim, try again, if that failed roll back again and ask the user to reduce the volume of application data open by closing views or documents or whatever before they try again. But yeah it’s a lot of engineering.
- gmueckl 7y agoSo you meant to say: ignore the possibility of this happening because it is extremely unlikely, too complex to handle and your time is better spent fixing much ore serious bugs that the users report. Silly engineering like what you suggest is a waste of time. Non-critical applications that are allowed to crash (every normal desktop app) would end up overengineered and unmaintainable. Embedded systems that must give guarantees about their reliability simplify system design drastically. One of the things going out the window first is dynamic memory allocation. It has too many failure modes.
- Crinus 7y ago> ask the user You just failed to create a toolbar button, how are you going to ask a user do something if you already failed to create a tiny UI element?
- bloak 7y agoThis is true, in my experience. If you keep your program fairly simple, implement it in plain C, avoid recursion, and keep these requirements in mind from the start, then it is possible to write a program that fails gracefully when it runs out of memory. Some old-school Unix daemons do this. But for most software it's impractical, so you might as well define your own xmalloc() that calls abort() if malloc() returns zero and at least have an immediate and obvious failure, rather than obscure memory leaks and data corruption.
- unilynx 7y agoAn exception-safe C++ program can do the same through a std::bad_alloc exception. If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine. (I usually connected other out-of-resource conditions such as pipe/socket/open failures to std::bad_alloc too. They're pretty comparable in effect and give a bigger chance of actually exercising those exception paths)
- lmm 7y ago> If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine. If you have a system that handles independent requests, it might be more robust to model each request as its own process, and then simply allow those processes to crash if they find themselves in an unexpected condition (and handle process crashes).
- unilynx 7y agoIt's certainly more robust, but a lot of overhead for what should be a very uncommon situation. In eg an application server, requests themselves may be independent, but still share a lot of cached data - and as long as locking overhead doesn't overwhelm you, a single process is still the fastest way to share data between (worker) threads.
- Vogtinator 7y agoIf you write code with memory exhaustion in mind, which mostly means not relying on allocations in the error handling paths, malloc fails are perfectly fine to handle.
- dooglius 7y agoIf you take a look at the Linux kernel, a hugely complex and sophisticated piece of software, you'll find that it gracefully handles allocation failures.
- cesarb 7y agoThe kernel's "graceful handling" of allocation failures includes killing unrelated processes in an attempt to free some memory.
- shereadsthenews 7y agoAlso just deadlocking. The kernel's behavior in the face of real memory pressure is actually quite poor. It's unsafe to run a Linux box out of memory.
- hazeii 7y agoDepends; for example one video application I've written buffers as many incoming frames in memory as possible; if malloc fails when taking the buffer from 8000 to 9000 frames it's not a problem; just carry on running with the current buffer size.
- imtringued 7y agoYou still have to gracefully shut down the application by closing file handles, aborting database transactions, etc.
- Crinus 7y agoThe OS will close the file handles and any database worth its salt should also abort the transaction automatically when the connection to it is lost before the transaction is finished.
- deleted 7y ago[deleted]
- shereadsthenews 7y agoWhen a memory control group runs out of space the process allocating just gets unilaterally killed with no notification. ulimit instead makes brk or mmap return a failure code, which the application could attempt to handle.
- asveikau 7y agoAnother thing to keep in mind, though 32 bit is less and less common over time, malloc would probably fail on a 32 bit process that is out of address space.
- TorKlingberg 7y agoRaspberry Pis almost always run 32 bit OSes, so that's a common place to encounter 32-bit.
- saagarjha 7y agoI had always thought they'd be running a 64-bit OS, given that the new hardware is 64-bit. TIL that Raspbian is still 32-bit!
- TorKlingberg 7y agoIt seems Broadcom never released the required binary blobs in 64-bit format. People love to hate on Intel, but the ARM SoC world is quite terrible about binary blobs.
- gmueckl 7y agoUnless you need the address space, there is little gain with a 64 bit OS. The wider pointers use more RAM and eat up more of the available memory bandwidth and caches. So not swotching to 64 bits is likely the best use if the hardware.
- saagarjha 7y agoIt’s a toss up, I think: actual results depend on the task at hand. 64-bit has the benefit of more, wider registers, which can speed up certain tasks.
- gmueckl 7y agoI stand corrected. I missed the fact that ARM also introduced new registers in AArch64. I thought that such a change only happened in amd64.
- lasthacker 7y agomalloc fails with a negative argument
- saagarjha 7y agomalloc takes a size_t, which is unsigned. A "negative" argument is just something that's absurdly large.
- NullPrefix 7y agoOn 64 bit systems it would result in a call to malloc asking for between 9.2 and 18.4 Exabytes.
- systemBuilder 7y agoA
- em3rgent0rdr 7y agoMalloc allocates virtual memory. The original article had to issue a correction at the end: "I was wrong about why malloc finally failed! @GodmarBack observes, in the comments, that x64 systems only have an address space of 48 bits, which comes out to about 131000 GB. So, on my machine at least, the malloc finally failed because of address space exhaustion."
- metrxqin 7y agoThat's incorrect, 2^48 bits = 262144GB actually.
- lifthrasiir 7y agoBut x86-64 divides the 48-bit address space into two halves, only one of which is available in the user mode.
- monocasa 7y agoTo be fair, there's nothing that says the higher half _has_ to be entirely kernel mode. Only that bits 63 through 48 have to be the same value. And Intel has a spec out for a PML5 page table, giving you 57 total virtual address bits.
- unilynx 7y agoThey forgot to mention the kernel gets the top half of the address space, leaving 2^47 for the process. And then it mostly adds up.
- dooglius 7y agoThe title here is just blatantly false; there are certainly some scenarios in which it won't fail, but many in which it will. The author is aware of this since he fixes the claim toward the end: > To clarify, the surprising behaviour malloc has does not mean we should ignore its return value. We just need to be careful because malloc returning successfully does not always mean that we can use the requested memory. It's worth pointing out that if you turn overcommit off (https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...) you will, in fact, get this guarantee
- chrisseaton 7y ago> does not always mean > if you turn overcommit off The author said ‘does not always mean’ so giving one condition in which it does doesn’t prove them wrong does it?
- dooglius 7y agoYou're right, based on the context of the post, I was interpreting that statement to make a broader claim than it was; I've modified my comment not to contradict it.
- twic 7y agoIt's a shame that turning off overcommit is done at the system level, rather than the malloc call level. Some applications, and perhaps some allocations within otherwise naive applications, might prefer to have an allocation fail early, where failure could be sensibly handled.
- mannykannot 7y agoI think that would ultimately have undesirable consequences. If this were an option on a malloc call, any such call would have to reserve the allocated memory at that point, reducing the usefulness of overcommit for other processes. This would set up a 'tragedy of the commons' scenario, where every application developer defensively uses this feature because other applications are using it.
- saagarjha 7y ago> Section III of this Phrack article is a down to earth description of how the glibc malloc implementation works, if you’re curious. This is probably outdated, given that it was written in 2009.
- mathieubordere 7y agopocorgtfo 0x18 contains a more recent glibc malloc exploit which goes into quite a bit of detail if you're interested.
- boomlinde 7y agoYou can disable memory overcommit in the kernel, before any of you start assuming you can drop your malloc checks.
- mehrdadn 7y agoTranslation: This is a euphemism for saying Linux blatantly violates the language specification with no remorse. (Yes, I realize Linux is the kernel, etc.)
- Liquid_Fire 7y agoWhich part of the specification does this violate?
- mehrdadn 7y agoLike, every part? It doesn't actually allocate the space requested, it doesn't indicate failure by returning NULL (or by any other reliable means for that matter)...
- Liquid_Fire 7y agoI had a look, and the C standard doesn't actually say anything about indicating failure. Here is all it says about malloc: > The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate. > The malloc function returns either a null pointer or a pointer to the allocated space. I think it's certainly debatable whether overcommitting while lazily allocating counts as actually allocating.
- mehrdadn 7y ago> I had a look, and the C standard doesn't actually say anything about indicating failure. > The malloc function returns either a null pointer or a pointer to the allocated space. Uhm, either malloc returns a pointer to the allocated space, or it returns null. I don't see what's so complicated about this. There's no provision for "return a non-null pointer to unallocated space".
- brianfen 7y agoThe malloc() function requests memory from the kernel using mmap(). If mmap() returns successfully, it means that the kernel is saying that the memory is allocated. So from malloc()'s perspective the memory allocation has been successful and it must return a non-null pointer to the caller. The standard does not require malloc() to distrust what the kernel said and try to actually write to that memory to double-check if there is any physical memory mapped to it or not. The standard also does not impose anything on the kernel. I see no section of the standard being violated here. Section 7.22.3.1: The order and contiguity of storage allocated by successive calls to the aligned_alloc, calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space. If the space cannot be allocated, a null pointer is returned. If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object. Section 7.22.3.4: The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate.
- philpem 7y agoI saw the title and thought "oh heck this isn't right at all"... Set VM Overcommit to zero on an embedded system with no swap (a Raspberry Pi will do nicely). Write a C program that malloc()'s all the RAM. Watch malloc start to fail when you hit the RAM limit and the kernel has dumped all the I/O cache it can.
- KirinDave 7y agoI think the author uses a hyperbolistic title, but if you read the article it s addressed that there are ways it can fail. The larger point is that very few users of malloc understand its semantics, and in fact you can't know exactly how malloc will behave without knowing things about the runtime configuration of the system (as opposed to the hardware availability as many people like to think the simple case is).
- antisemiotic 7y agoAnd then, some day, your code gets compiled for Windows under mingw/msys.
- aetherspawn 7y agoHow does one guarantee that allocated memory is real? Can you easily wrap malloc to zero/poke the memory in a way that catches all exceptions and guarantees yay or nay ?
- anilakar 7y agoJens Gustedt suggested the following: memset(malloc(size), 0, 1) The program will crash if malloc returns NULL or the memory is not writable.
- hyperman1 7y agoAnd then memory deduplication comes along, finds out a huge number of identical pages (all zero) and decides to deduplicate them. So if you want your memory to stay real, I suppose you'd better fill it with random data or something.
- shereadsthenews 7y agoYou must use mlock call if you want to guarantee that no later action can suffer a major page fault.
- tus87 7y agoMalloc is just mmap so how about: ENOMEM The process's maximum number of mappings would have been exceeded. This error can also occur for munmap(), when unmapping a region in the middle of an existing mapping, since this results in two smaller mappings on either side of the region being unmapped.
- deleted 7y ago[deleted]
- JasuM 7y agoWhat happens if you really run out of physical memory (including swap) after overcommitting? Does your process get a signal, or will OOM killer just run without notifying the process that triggered the condition?
- gmueckl 7y agoIg you try to allocate more than the system is willing to overcommit (e.g. a huge block all at once), malloc will fail. But if phyaical memory gets exhausted by accessing previously allocated pages, the OOM killer will evebtuslly come around and kill processes without signalling. Signal handlers could still make thenprocess (unknowingly) request more memory, so there is 0 guarantee that a handler could even run successfully.
- JasuM 7y agoOh yeah, I didn't think of that. I wonder if you could write a signal handler carefully to not allocate any memory, stack or otherwise, or is some return address or an internal structure being allocated transparently...
- gmueckl 7y agoJust trying to allocate stack space for the signal handler may cause the stack to spill into a new page. That is absolutely out of anybody's control. And if that new page cannot be provided, it's game over.
- JasuM 7y agoMakes sense. I was thinking that maybe signal handlers could use the regular stack of the thread, but that would of course make everything fall down if the "real" code would write to the stack before updating the stack pointer.
- Kenji 7y agoI would assume you get a segfault if you try to access more than a few of those 130000 1-GB blocks.
- pornel 7y agoThis assumption lead to Rust's standard library not having a way to catch allocation failures (which is only now being rectified, and only partially). It's very Linux-centric and presumes a certain config+usage pattern. Not true on Windows. Not quite true on macOS. Not true in WASM. Definitely not true on embedded platforms.
- mmillin 7y agoIs there somewhere to read about how rust is tackling this problem?
- cesarb 7y agoIt seems to be described in this Rust RFC: https://github.com/rust-lang/rfcs/blob/master/text/2116-alloc-me-maybe.md https://github.com/rust-lang/rfcs/blob/master/text/2116-allo...
- steveklabnik 7y agoA short overview: 1. Rust the language knows nothing about allocation. If you care about this behavior, it mostly limits the code of others' that you can use, but you can always write your own versions of things that respect fallible allocations. 2. Rust's standard library assumes memory is infallible. This is partially because it's a good default, and partially because our allocator API was not ready yet. 3. We've been working on the allocator API. 4. We have a rough plan for parameterizing data structures over allocators. 5. If this topic is of interest to you, https://github.com/rust-lang/wg-allocators https://github.com/rust-lang/wg-allocators is where to get involved.
- shereadsthenews 7y agoAttempting to catch allocation failures is the stupidest feature of C++ and the main reason that people turn off exceptions in that language.
- brynet 7y agoThe malloc(3) family will absolutely fail on OpenBSD, for many reasons: login.conf/ulimits, calloc(3) for when integer overflow is detected (nmemb * size), same for OpenBSD's reallocarray extension. And yes, because the system was unable to satisfy the requested allocation. Linux overcommit, and developer mindset is a detriment to software quality and portability.
- jiveturkey 7y agoThat's a needlessly confrontational statement. Yet I find myself in agreement. Sigh.
- jvanderbot 7y agoWhat bothers me is that I read this train wreck without any red flags until I saw his correction at the end. Even the headline was wrong given the article, which itself was wrong. I really should have coffee before HN
- dragontamer 7y agoWhat exactly is "wrong" about the article? Linux's Overcommit behavior is non-obvious to many programmers. Its one of those issues that very few programmers I've come across in the workplace understand properly. This blogpost properly understands the issues associated with Overcommit, and have done some preliminary investigations that describe the behavior. Its a really good blogpost. The general point of the blogpost is that "Malloc Fails due to address space exhaustion more often than actual memory-exhaustion". Because actual memory exhaustion causes OOM killer code to be run... and OOM killer is a non-obvious case of the Linux kernel.
- 95014_refugee 7y agoWhat's "wrong" about the article is that the author has no idea what malloc(3) does, or about the wide range of execution environments C programs find themselves in and the different ways that the API maps to those environments. More generally, the article does nothing to educate the public or move the "debate" about overcommit in any sort of useful direction.
- shereadsthenews 7y agoWhat's wrong with the article is that malloc is not even a feature of Linux. There's mmap and brk, both of which have documented failure modes.
- dragontamer 7y agoMost typical programmers will be using malloc, or some mechanism built on top of malloc (C++ new). MMap and brk are useful for certain situations (explicitly getting Huge Pages), but I don't think that the typical programmer necessarily needs to know about those. Since glibc malloc is built on top of mmap and brk, I think your distinction is mostly academic. For any programmer using Linux and glibc... malloc's failure mode IS mmap and brk failure modes.
- Paul-ish 7y agoWhen I use Python sometimes I run into a MemoryError when working with large datasets. How does the Python runtime know I am out of memory if the kernel won't tell it. Does it try a write and catch the signal?
- amelius 7y agoPerhaps any write can fail, and if one does then it triggers some code which frees some pre-reserved space which allows the interpreter to continue, and properly unwind the stack.
- lalaithion 7y agoI believe that the python interpreter creates a MemoryError at startup, and throws that when you run out of memory.
- amelius 7y agoOk, but while unwinding a try/except block, the interpreter might need some extra memory, I suppose, depending on the code in the except clause.
- cesarb 7y agoIf your large dataset leads to a single huge memory allocation, the kernel will fail the allocation even though overcommit is enabled. Early versions of libvpx had a similar issue, where it allocated a huge amount of memory at once (actually, told the dynamic linker to do so) which failed on low-memory machines (https://bugzilla.redhat.com/show_bug.cgi?id=600663 https://bugzilla.redhat.com/show_bug.cgi?id=600663 "libvpx.so.0 fails to load on low-memory machines due to ridiculously oversized .bss").
- edoo 7y agoI always assume malloc success and that if it were to return null the system is screwed anyway and the app will just abort. I write a lot of 'critical' C/C++ now on embedded systems with mere KB of RAM and just never use the heap ever.
- antirez 7y agoInteresting topic with many things to say about, but wrong content. The point is that in most C programs, it is not worth to handle OOM errors, because what you can do during OOM is of very little value, on the other hand handling OOM correctly is very hard. However you can't do this in libraries, because you don't know how the library is going to be used. So for instance in order to make my Radix tree library resistant to OOM failures, I had to write a specific fuzzy test that used a malloc failing with a given probability, and check if the tree is sane after some stress-work with such malloc. In general the complexity of handling OOM in complex programs that deal with complex data structures is not often recognized.
- monocasa 7y agoWhen writing C libraries, it's good form to be able to init your instance with allocation and logging function pointers. That lets you play nice with most any env you're being pulled into, and gives your consumers an obvious place for nice hooks for debugging.
- antirez 7y agoYep, that's what I do in order to be able to fuzz test with an OOM returning malloc() in my lib. Agreed on the fact it's good form.
- systemBuilder 7y agoSeems like a needless feature, overcommit, "oh let's make our app faster by pre allocating 2GB of RAM!" says the naive programmers, "Oh let's make our operating system more powerful by allowing overcommit.". Overall, you moved the allocation delay from the app to the OS, which has much less end to end knowledge of how to do it effectively, overcomplicated both layers, achieved practically nothing overall, like a dog chasing it's tail, except now both layers have more garbage code, so it's like an obese dog chasing it's tail!
- kazinator 7y agoYou can arrange for malloc to fail. If overcommit is disabled with the right sysctl parameters, then mmap will fail if there isn't enough physical memory to materialize the entire mapping. Even under overcommit, very large malloc requests that translate directly to large mmap requests can fail with a null return.