6 ms·
Some bits on malloc(0) in C being allowed to return NULL
- bobmcnamara 1y agoAges ago I worked with a system where malloc(0) incremented a counter and returned -1. free(-1) decremented the counter. This way you could check for leaks :p
- o11c 1y agoNoncompliant, since `malloc(0)` is specified to return a unique pointer if it's not `NULL`. On most platforms an implementation could just return adjacent addresses from the top half of the address space. On 32-bit platforms it doesn't take long to run out of such address space however, and you don't want to waste the space for a bitmap allocator. I suppose you could just use a counter for each 64K region or something, so you can reuse it if the right number of elements has been freed ...
- LPisGood 1y agoNoncompliant, but what could this reasonably impact?
- bobmcnamara 1y ago> Noncompliant, since `malloc(0)` is specified to return a unique pointer if it's not `NULL`. I know I've seen that somewhere, but may I ask what standard you're referring to?
- masfuerte 1y agoIt's POSIX. > Each [...] allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space. If the space cannot be allocated, a null pointer shall be returned. If the size of the space requested is 0, the behavior is implementation-defined: either a null pointer shall be returned, or the behavior shall be as if the size were some non-zero value, except that the behavior is undefined if the returned pointer is used to access an object. https://pubs.opengroup.org/onlinepubs/9799919799/functions/malloc.html https://pubs.opengroup.org/onlinepubs/9799919799/functions/m...
- MaxBarraclough 1y agoNot just POSIX, also the ISO C standard itself. https://en.cppreference.com/w/c/memory/malloc https://en.cppreference.com/w/c/memory/malloc
- masfuerte 1y agoThat doesn't say the pointer has to be unique.
- jcranmer 1y agocppreference isn't the standard, and while the text they write looks like it's the same verbiage that would be authoritative, it's not. (And there's some criticism of it from standards committee members in that regard). The current C standard text says: > The order and contiguity of storage allocated by successive calls to the aligned_alloc, calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it can be assigned to a pointer to any type of object with a fundamental alignment requirement and size less than or equal to the size requested. It can then be used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space. If the space cannot be allocated, a null pointer is returned. If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned to indicate an error, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object. So yeah, the allocations are required to be unique (at least until it's free'd).
- bobmcnamara 1y ago> Each such allocation shall yield a pointer to an object disjoint from any other object. Phrasing could be slightly more clear to prevent someone from making the argument that -1 is disjoint from all objects as it does not point to an object
- o11c 1y agoPointers are frequently used as keys for map-like data structures. This introduces collisions that the programmer can't check for, whereas NULL is very often special-cased.
- bobmcnamara 1y ago> Noncompliant, since `malloc(0)` is specified to return a unique pointer if it's not `NULL`. I know I've seen that somewhere, but may I ask what standard you're referring to? If I recall correctly, this was an archaic stackless microcontroller. The heap support was mostly a marketing claim.
- jmgao 1y agoC89: https://port70.net/%7Ensz/c/c89/c89-draft.html https://port70.net/%7Ensz/c/c89/c89-draft.html If the size of the space requested is zero, the behavior is implementation-defined; the value returned shall be either a null pointer or a unique pointer.
- f1shy 1y agoIsn’t -1 basically 0xffff which is a constant pointer? What am I missinterpreting?
- comex 1y agoIf you call malloc(0) multiple times (without freeing in between) and get -1 each time, then the pointer is not unique.
- deleted 1y ago[deleted]
- bobmcnamara 1y agoBut do we need a unique pointer or merely a pointer that is disjoint from all objects?
- david-gpu 1y agoAs per the specification, it has to be a unique pointer. Being tasked to implement a specification typically means having to pass extensive conformance tests and having to answer for instances of noncompliance. You soon learn to follow the spec to the letter, to the best of your abilities, unless you can make a strong case to your management for each specific deviation.
- j1elo 1y agoOh but no worries with compliance, it always returned a newly created -1, never repeating the same one!
- hun3 1y agoMy next malloc(3) is returning NAN.
- fredfish 1y agoI've always considered NaN too definitive for general industry languages like C, JS or Cobol where not even physics with calculus should be assumed. Maybe its ok for languages that at least expect math for engineers like Fortran or up.. How about we call it "Maybe a Number" and since equality can't work for it we still need a separate way to ask like: Math.whoIsTheMaN(me)
- sgerenser 1y agoI might be missing something, but how does this help in checking for leaks? I mean, I guess you could use it to check for leaks specifically of 0-sized allocations, but wouldn’t it be better just to return NULL and guarantee that 0-sized allocations never use any memory at all?
- bobmcnamara 1y agoAt the end of main, if the count wasn't balanced, then you knew you had a mismatch between malloc()/free(). If malloc() had returned a real pointer, you'd have to free that too. > wouldn’t it be better just to return NULL and guarantee that 0-sized allocations never use any memory at all? Better: takes less memory Worse: blinds you to this portability issue.
- Someone 1y ago> At the end of main, if the count wasn't balanced, then you knew you had a mismatch between malloc()/free(). A mismatch between malloc(0) and free(-1). You’d know nothing about calls to malloc with non-zero sizes.
- sgerenser 1y agoYeah, exactly, that’s my point. How many programs have memory leaks limited to (or even just materially affected by) 0-sized allocations? I’d have to imagine its a very small minority.
- bobmcnamara 1y agoThey're uncommon for sure. In the past they've been an issue for me on constrained systems where they can frag the heap as badly as any other long lived allocation.
- bobmcnamara 1y agoThose are identifiable by the end state of the heap not being empty.
- sweetjuly 1y agoDoes this work in practice? Now you have a bunch of invalid but non-NULL pointers flying around. NULL checks which would normally prevent you from accessing invalid pointers now will pass and send you along to deref your bogus pointer. Even hacking the compiler to treat -1 as equal to NULL as well wouldn't work since lots of software won't free NULL-like pointers.
- bobmcnamara 1y ago> NULL checks which would normally prevent you from accessing invalid pointers now will pass and send you along to deref your bogus pointer. Oddly, this is bog standard implementation specific behavior for standard C - caller accessing any result of malloc(0) is undefined behavior, and malloc(0) isn't required to return NULL - the reference heap didn't, and some probably still don't.
- sweetjuly 1y agoAh, that's my bad. Another day, another UB :)
- bobmcnamara 1y agoLike swimming in a bucket of rusty knives :)
- carra 1y agoNot the best choice to begin the title with "some bits" in this context. My mind was trying to understand this sentence in a completely different way...
- AaronDinesh 1y agoWhy should it be allowed to return a valid pointers anyways? Surely it should always return NULL?
- Joker_vD 1y agoFor instance, because you are prohibited from passing NULL to e.g. memcpy and lots of other library functions from memory.h/string.h, even when you explicitly specify a size of 0. Another use was to use it to mint unique cookies/addresses, but malloc(1) works for this just as well.
- TZubiri 1y agoMmmmh, cookies
- snickerbockers 1y agoIt's not a valid pointer because you can't use the indirection operator on it. Returning a value other than NULL makes sense because an allocation of size zero is still an allocation. Additionally the actual amount of memory malloc allocates is implementation-defined so long as it is not less than the amount requested, but accessing this extra memory is undefined behavior since processes don't know if it exists or not. a non-NULL return could be interpreted as malloc(0) allocating more than zero bytes. Some implementations don't actually perform the allocation until theres a pagefault from the process writing to or reading from that memory so in that sense a non-NULL return is valid too. I'd argue that malloc(0)==NULL makes less sense because there's no distinction between failure and success. The only real problem is specifying two alternate behaviors and declaring them both to be equally valid.
- cjensen 1y agoThere are three reasonable choices: (a) return the null pointer (b) return a valid unique pointer and (c) abort(). The point of the original C Standard was to make rules about these things AND not break existing implementations. They recognized that (a) and (b) were in existing implementations and were reasonable, and they chose not to break the existing implementations when writing the standard. This is similar to the extremely unfortunate definition of the NULL macro. There were two existing styles of implementation (bare literal 0 and (void *) 0) and the Standard allows either style. Which means the NULL macro is not entirely safe to use in portable code.
- eesmith 1y agoI maintained a program which failed on, as I recall, AIX (mentioned in the essay) because malloc(0) returned NULL. It's been 30 years so I've forgotten the details. My solution was to always allocate size+1 since memory use was far from critical.
- randomNumber7 1y agoI never had the use case to allocate 0 bytes of memory. If I would allocate 0 bytes of memory and get a pointer to it, I wouldn't care what the value of the pointer is since I am not allowed to dereference it anyways. But then again, why would I allocate 0 bytes of memory?
- bobmcnamara 1y agoSometimes it shakes out simpler for a generic container. Ex: a vector using only a counter and pointer - you can use realloc() with fewer pointer validity checks.
- corysama 1y agoSome folks used it as a per-run, internal GUID generator. Not saying that's a good use. But, it was done.
- Lvl999Noob 1y agoCan someone tell me a usecase where you want multiple allocations of size 0, each one with a unique address, and each one unique from any other allocation (hence necessarily removing that pointer from being allocated to anything else) but can't use malloc(1) instead? I think it would be much better if malloc(0) just returned 1 or -1 or something constant. If the programmer needs the allocation to have a unique address, they can call malloc(1) instead.
- hansvm 1y agoIt's occasionally useful to want multiple allocations of size 0, each one with a valid address -- generic containers parsing something as a some sort of sequence object and you want all code interacting with it to do something valid. I'd be hard-pressed to see where you'd need those to be unique though. Basically any integer should be fine.
- xenadu02 1y agoBecause zero-size types exist which you might want to take the address of. Possibly as a result of macro substitution or templating mechanism that only appears in certain build configurations. It means you don't need a bunch of special-case handling if one out of 27 types ends up with zero size in some situation. It just all works the same way. Especially the unique address part because that would be an annoying source of difficult to track bugs.
- Lvl999Noob 1y agoYes. I believe zero sized types should be possible and they should all have the same address. Trying to deref the pointer is UB right away because you do not have the byte under that pointer. As it is, the malloc implementation now needs special casing for 0 sized allocations and different implementations special case it differently. C is supposed to be low level so surface this confusion up. Let the programmer decide if they want a unique address and reserve a byte or a non unique one with no overhead.
- zzo38computer 1y ago
- tptacek 1y agoI get the complexity of the standards issue here, but if you cared about this, wouldn't you just wrap malloc with something trivial that provided the semantic you wanted to depend on (NULL or some sentinel pointer).
- a-dub 1y agowould be interesting to see if there's a difference in how the 0-page is handled in systems under this condition...