13 ms·

Once upon a time, memory allocators made sense

142 points by dchest 11y ago

bjornsing 11y ago
To me it seems more reasonable to draw the insanity line at returning NULL for a successful zero byte allocation. (While it's true that dereferencing a pointer to a zero byte allocation should probably trigger undefined behavior that doesn't mean that it shouldn't be possible to do the != NULL test to see if the allocation was successful.)
- akeruu 11y ago
  To me, the insanity is trying to allocate zero byte
  - bjornsing 11y ago
    Agree, but that's not the allocator's sanity though.
  - zAy0LfpBZLC8mAC 11y ago
    That's because you think of it as a special case, which it isn't. If you want to store a string that's n bytes long, you need to allocate n bytes, so a natural thing to write is, for example dst.len=strlen(src);dst.buf=malloc(dst.len); --it's insanity if you need to add a special case for strings of a specific length.
    - tbirdz 11y ago
      A standard C string will never be 0 bytes. There is the zero terminator at the end of the string, so all strings will be at least 1 byte large.
      - carussell 11y ago
        The example is clearly forgoing the use of standard C strings for a char array tagged with the length.
    - acqq 11y ago
      You don't have to call malloc directly in the second assignment. If you depend on special handling of null without the "if" on invocation you can have your own function for that.
- gpderetta 11y ago
  As it guaranteed to work by the appropriate standards, there are programs that use zero bytes allocations to get a process wide unique cookie. edit: turns out that no, neither C11 nor C++11 guarantee that, it was probably historical behaviour.
david-given 11y ago
Plus, of course, a lot of modern systems use memory overcommit --- if you ask the OS for memory, it gives you uninitialised address space, and it only allocates pages of physical RAM the first time you touch each page of address space. This has two effects: (a) malloc() never returns NULL. It always returns a valid address, even though your system may have out of memory. (b) by the time the kernel finds out that it's run out of physical pages, your process is already trying to use that memory... which means there's no way for your process to cope gracefully. You have to trust the kernel to do the right thing (either to scavenge a page from elsewhere, or to kill a process to make space). If you're very lucky, it'll send you a SIGSEGV...
- kbart 11y ago
  Hm, do you have any test/references on that? I always thought that in such cases system simply swaps memory to free some RAM.
  - amadvance 11y ago
    Google for Linux OOM Killer (Out Of Memory Killer). It's really annoying, your application may get killed without any way to react, even to just print a "Low memory" error.
    - feld 11y ago
      The entire concept of the Linux OOM Killer is asinine
  - andreasvc 11y ago
    It is a well known design feature (I would call it a flaw) of Unix/Linux which makes forking processes with copy-on-write memory possible. You can run out of both physical memory and swap space. Or your system is swapping is so heavily that, for all practical purposes, the system becomes unusable.
    - nickpsecurity 11y ago
kbart 11y ago
I don't get it. For me it's always a simple logic: if you didn't have anything (0 bytes) to allocate/reallocate you simply don't do that. Period. It's like division by zero -- you always have to check if divisor is not equal to zero; if it was -- skip division altogether.
- zAy0LfpBZLC8mAC 11y ago
  Except the semantics of a zero-length buffer are perfectly well-defined, while the semantics of division by zero are not defined at all in the types commonly used in computing. A proper comparison therefore would be addition, maybe, which is also perfectly well-defined for zero operands. Do you special-case "zero addition" in your code? if(a==0){if(b==0){x=0;}else{x=b;}}else{if(b==0){x=a;}else{x=a+b;}} ?
  - kbart 11y ago
    I'd argue that zero-length buffer is far from "perfectly well-defined". Quoting C99: " If the size of the space requested is zero, the behavior is implementation- deﬁned: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object." To maintain my sanity as a C programmer, I steer away from "implementation specific behavior" as much as possible, thus I prefer checking if buffer size is non-zero before allocating.
    - zAy0LfpBZLC8mAC 11y ago
      I am not talking about C, but about the abstract concept. C's implementation is obviously kindof broken. Just as C would be broken if the behaviour of "zero additions" was somehow "implementation defined". That doesn't mean that something is fundamentally wrong with adding zero, it just means that you have to work around a broken language.
      - kbart 11y ago
        The topic is about C and I never said that such behavior is normal or anticipated. It's one of many cases where C standard lacks concreteness and we can only workaround such issues. In a perfect world, many things would be different.
jheriko 11y ago
erm... allocating zero was never a good idea. it had varying behaviours on varying platforms for as long as i could remember and was one of those special cases to avoid. the allocators didn't necessary even meet the alignment specification either... even if the documents said they did. i remember reading that malloc on windows would return things on 8-byte boundaries only to find pointers ending in 4 (rather than 0 or 8) coming out of it... and as many have pointed out returning null may or may not happen in a number of situations. i've seen it happen enough times when i ask for too much memory to believe that it is a useful fail case to check for...
- kazinator 11y ago
  realloc(ptr, 0) special case which (on some C libraries only) means the same thing as free(ptr) and not to realloc the object to zero size. That's what the author's resize function expected, which is why his patch is to use free.
  - fanf2 11y ago
    C libraries do not conform to the standard if they do not free in this case.
    - masklinn 11y ago
      That requirement was present in C89 and removed in C99, which also added that a 0-sized allocation doesn't have to return a null pointer (whether done by malloc, calloc or realloc)
      - kazinator 11y ago
        Dang, is that right? See, I checked C99 when I posted the comment. I don't know where my old copy of C90 is, but I was mildly surprised. I had that WTF feeling "I could have sworn it was supposed to work that way! In any case, if I'm compiling with -ansi or -std=c90 on a GCC-based platform that is hosted, the library had better behave in the C90 conforming manner. Only if if -std=c99 is used should it take the liberty with realloc(ptr, 0) being like malloc(0).
antirez 11y ago
The fact when you reallocate you mean to trow the data away does not make sense... In general the article seems honestly a bit confused in what it states.
- cperciva 11y ago
  realloc can change the size of a memory allocation and return the same pointer.
  - antirez 11y ago
    What I mean is that, when you say: > Unfortunately, some people are not sane, and have decided that it's equally valid for realloc(p, 0) to return NULL meaning-failure and not free p. What you want is the following semantics: char *my_old_vector = ... ; /* We created / allocated in some way. */ add_important_stuff_to(my_old_vector); char *new_vector = realloc(my_old_vector,size+100); if (new_vector == NULL) { /* I still have the old vector and can recover. */ }
carussell 11y ago
> So what happens if realloc(p, 0) fails? It's going to return NULL, since that's what realloc does if it fails; but should it also free the allocation p? The sane answer is yes If realloc(p, 0) is a synonym for free(p), what does it mean for it to "also free the allocation p" in the event of failure? Unpacking it, the statement seems to be saying the same thing as, "if it fails, then what it should do is to not fail".
- cperciva 11y ago
  Freeing a memory allocation is guaranteed to never fail. Allocating zero bytes can fail (with dumb C libraries). The question is "if realloc fails to allocate zero bytes, should it still free the original allocation".
  - carussell 11y ago
    I'm still not clear on this. What does "allocating zero bytes" mean, if not "free the original allocation"? Keep the malloc metadata structures around but update their size to 0?
    - cperciva 11y ago
      Some systems handle a zero-byte allocation request by allocating a unique non-NULL pointer which cannot be dereferenced. If they run out of unique pointers, they can fail to allocate zero bytes.
  - pcwalton 11y ago
    > Allocating zero bytes can fail (with dumb C libraries). They might not be dumb; they might be forced into it to maintain compatibility with buggy programs and/or configure scripts (e.g. [1]). [1]: http://www.openwall.com/lists/musl/2013/01/15/1 http://www.openwall.com/lists/musl/2013/01/15/1
    - cperciva 11y ago
      Unfortunately, as I wrote in the commit message which spawned this thread, this just encourages people to write more buggy programs.
coldpie 11y ago
The REAL problem here is C's garbage way of dealing with errors. NULL is overloaded both to be a valid pointer and an error signal. In the author's ideal world, how would a malloc failure be communicated?
- scott_s 11y ago
  Null is not a valid pointer, though. It's a valid pointer value, but it's undefined to dereference such a pointer. If the purpose of malloc is to return an address to usable memory, then null it outside of that range. Values outside of the legal range of a function are fine for error conditions. Because of that, null seems like a reasonable return value for error on malloc: I could not allocate memory for you, which I indicate by returning an address that you are not supposed to dereference. However, I do agree it would be better if there was some other kind of error condition, preferably something like the option type from various other languages, or just an easy way to return multiple values like in Go. But C is not that language.
  - jzwinck 11y ago
    C can return a struct from a function. That's multiple return values.
    - scott_s 11y ago
      Yeah, but it's cumbersome because you have to define that struct elsewhere, and C does not have elegant mechanisms for destructing in-place.
      - ratboy666 11y ago
        How about #include <stdio.h> typedef struct mystruct { int x; char *p; } mystruct; mystruct func(int n) { return (mystruct) { n, NULL }; } int main(void) { mystruct v = func(1); printf("%d\n", v.x); printf("%d\n", func(2).x); return 0; } Works fine with gcc 5.3.1. Multiple destructuring is (maybe) problematic, but if you really want this.
asveikau 11y ago
This is way too hyperbolic. I don't know what specific "once upon a time" period is being described. However malloc(0), realloc(NULL, x), and realloc(y, 0) has been a portability headache for as long as I have been writing C. None of this is all that surprising.
- klodolph 11y ago
  The behavior of realloc(NULL, x) is always equivalent to malloc(x) according to the C standard. But yes, I never remember a time when malloc(0) or realloc(y, 0) played nice.
  - asveikau 11y ago
    > The behavior of realloc(NULL, x) is always equivalent to malloc(x) according to the C standard. AFAIK if you try to do this with even a fairly recent Microsoft libc it won't work. I haven't checked the standards and history on this one, but keep in mind it's been only a recent development that MS gives a crap about C99. The idea that C89 did not mandate this makes sense to me because I remember now-obsolete Unixes not liking this either. [Edit: The MS documentation claims that it supports realloc(NULL, x) going back quite a while. I know I've been bitten by it not doing that in the current century, however...]
    - masklinn 11y ago
      > The idea that C89 did not mandate this makes sense to me because I remember now-obsolete Unixes not liking this either. C89 did mandate this: > If ptr is a null pointer, the realloc function behaves like the malloc function for the specified size. C89 also mandated that realloc(ptr, 0) be equivalent to free(ptr), which was removed in C99: > If size is zero and ptr is not a null pointer, the object it points to is freed.
acqq 11y ago
What I don't understand is why some programmers invest so much energy in using realloc. When you know that for most cases realloc actually does a new alloc and copy of the existing content, but has the corners of undefined or unwanted behavior, why not doing it in your own function in a way that you fully (for your needs) control?
- klodolph 11y ago
  You certainly could wrap realloc(), but besides the bizarre/unnecessary behavior with realloc(ptr, 0), it behaves exactly as you would expect: on failure it returns NULL and doesn't touch your pointer, on success it returns the new pointer. It doesn't seem like it requires any more "energy" than using malloc() and programming in C to begin with.
- acconsta 11y ago
  For large allocations, realloc can remap virtual memory instead of doing a naive copy: http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-and-laziness/ http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-a...
  - acqq 11y ago
    That's exactly when I'd use realloc. Looking who wrote the text, I also respect cperciva and believe he must have some good reasons and I'd be glad read which use cases he had, more than just "what's wrong with different reallocs." Because I'm not surprised that the corner cases aren't to everybody's (or even anybody's) satisfaction. It's C. Less is more and all that.
  - cperciva 11y ago
    And for small allocations, realloc can expand or shrink an allocation in-place, because it knows where it is placing other memory allocations.
    - acqq 11y ago
      Yes. If the allocation block is 16 bytes for example, the string growing from 10 to 11, 12, 13 etc could still be on the same place. But in which use cases are frequent reallocs actually needed, so much that you can recognize the performance impact? I'd really like to know, as I personally never had such problems. When the single allocations were too expensive I've just used some kind of memory pool. For small stuff realloc is still more expensive than just a few instructions on average when some pool is used.
kazinator 11y ago
The fix is to wrap malloc and friends with your own routines. You can have it so that "my_malloc" always returns NULL on a zero length object. Or, if you prefer, so that it always returns a unique pointer: void *my_malloc(size_t size) { return size ? malloc(size) : 0; /* option A */ } void *my_malloc(size_t size) { return malloc(size ? size : 1); /* option B */ } And you can make my_realloc have the nice freeing behavior: void *my_realloc(void *ptr, size_t size) { return (ptr && size) ? realloc(ptr, size) : !ptr ? my_malloc(size) : (free(ptr), 0); } Note how we still carefully implement a parallel requirement to the one in ISO C. Namely that my_realloc(NULL, size) calls my_malloc(size), regardless of size, just like realloc(NULL, size) is required to be equivalent to malloc(size). We want all the routines in this allocator to be drop-in replacements such that any code which is written to the ISO C allocator spec can be blindly retargetted to use them. Pretty much any serious C program, if indeed it doesn't have an entire allocator of its own, at least wraps the standard one, for the sake of more uniform behaviors, as well as easy retargettability to embedded scenarios. (Not only embedded machines, but say, embedding in a larger application, where you are told "you must use this table of funtion pointers as your allocator" and it doesn't quite look like malloc: zero allocations are not allowed, realloc is missing, ...) In this particular program, realloc is wrapped under a function called resize; resize had to be patched not to rely on realloc(nonnull, 0), but users of resize don't have to change. The programmer doesn't have to look for fifty uses of resize to fix them. However, it's worth it to wrap the functions at a lower level anyway, with an identical API. The C library is far from perfect. Do not expect consistency, completeness, beauty, symmetry and so on. There are worse things in it than realloc(ptr, 0) not behaving in the neat way that you would like. Oh such as: isalpha(str[42]); // str is char array being undefined behavior if char happens to be signed, and str[42] is negative, because isalpha takes an int argument which is expected to hold a positive byte value [0, UCHAR_MAX), and not char value. Whoooops!
- klodolph 11y ago
  You can make my_realloc() simpler: void *my_realloc(void *ptr, size_t size) { return size ? realloc(ptr, size) : (free(ptr), 0); } According to the C standard, realloc(NULL, size) is guaranteed to be equivalent to malloc(size). From n1548 7.22.3.5 para 3: > If ptr is a null pointer, the realloc function behaves like the malloc function for the specified size. Also, free(0) is well-defined. From n1548 7.22.3.3 para 2: > If ptr is a null pointer, no action occurs.
theseoafs 11y ago
I don't know if the distinction between which behaviors are "sane" and "insane" is a good one here. Evidently the only insane choice is to rebuff the standard in your implementation of the core C memory management functions.

142 points by dchest 11y ago

To me it seems more reasonable to draw the insanity line at returning NULL for a successful zero byte allocation. (While it's true that dereferencing a pointer to a zero byte allocation should probably trigger undefined behavior that doesn't mean that it shouldn't be possible to do the != NULL test to see if the allocation was successful.)

akeruu 11y ago
To me, the insanity is trying to allocate zero byte
- bjornsing 11y ago
  Agree, but that's not the allocator's sanity though.
- zAy0LfpBZLC8mAC 11y ago
  That's because you think of it as a special case, which it isn't. If you want to store a string that's n bytes long, you need to allocate n bytes, so a natural thing to write is, for example dst.len=strlen(src);dst.buf=malloc(dst.len); --it's insanity if you need to add a special case for strings of a specific length.
  - tbirdz 11y ago
    A standard C string will never be 0 bytes. There is the zero terminator at the end of the string, so all strings will be at least 1 byte large.
    - carussell 11y ago
      The example is clearly forgoing the use of standard C strings for a char array tagged with the length.
  - acqq 11y ago
    You don't have to call malloc directly in the second assignment. If you depend on special handling of null without the "if" on invocation you can have your own function for that.
gpderetta 11y ago
As it guaranteed to work by the appropriate standards, there are programs that use zero bytes allocations to get a process wide unique cookie. edit: turns out that no, neither C11 nor C++11 guarantee that, it was probably historical behaviour.

david-given 11y ago

Plus, of course, a lot of modern systems use memory overcommit --- if you ask the OS for memory, it gives you uninitialised address space, and it only allocates pages of physical RAM the first time you touch each page of address space. This has two effects: (a) malloc() never returns NULL. It always returns a valid address, even though your system may have out of memory. (b) by the time the kernel finds out that it's run out of physical pages, your process is already trying to use that memory... which means there's no way for your process to cope gracefully. You have to trust the kernel to do the right thing (either to scavenge a page from elsewhere, or to kill a process to make space). If you're very lucky, it'll send you a SIGSEGV...

kbart 11y ago
Hm, do you have any test/references on that? I always thought that in such cases system simply swaps memory to free some RAM.
- amadvance 11y ago
  Google for Linux OOM Killer (Out Of Memory Killer). It's really annoying, your application may get killed without any way to react, even to just print a "Low memory" error.
  - feld 11y ago
    The entire concept of the Linux OOM Killer is asinine
- andreasvc 11y ago
  It is a well known design feature (I would call it a flaw) of Unix/Linux which makes forking processes with copy-on-write memory possible. You can run out of both physical memory and swap space. Or your system is swapping is so heavily that, for all practical purposes, the system becomes unusable.
  - nickpsecurity 11y ago

kbart 11y ago

I don't get it. For me it's always a simple logic: if you didn't have anything (0 bytes) to allocate/reallocate you simply don't do that. Period. It's like division by zero -- you always have to check if divisor is not equal to zero; if it was -- skip division altogether.

zAy0LfpBZLC8mAC 11y ago
Except the semantics of a zero-length buffer are perfectly well-defined, while the semantics of division by zero are not defined at all in the types commonly used in computing. A proper comparison therefore would be addition, maybe, which is also perfectly well-defined for zero operands. Do you special-case "zero addition" in your code? if(a==0){if(b==0){x=0;}else{x=b;}}else{if(b==0){x=a;}else{x=a+b;}} ?
- kbart 11y ago
  I'd argue that zero-length buffer is far from "perfectly well-defined". Quoting C99: " If the size of the space requested is zero, the behavior is implementation- deﬁned: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object." To maintain my sanity as a C programmer, I steer away from "implementation specific behavior" as much as possible, thus I prefer checking if buffer size is non-zero before allocating.
  - zAy0LfpBZLC8mAC 11y ago
    I am not talking about C, but about the abstract concept. C's implementation is obviously kindof broken. Just as C would be broken if the behaviour of "zero additions" was somehow "implementation defined". That doesn't mean that something is fundamentally wrong with adding zero, it just means that you have to work around a broken language.
    - kbart 11y ago
      The topic is about C and I never said that such behavior is normal or anticipated. It's one of many cases where C standard lacks concreteness and we can only workaround such issues. In a perfect world, many things would be different.

jheriko 11y ago

erm... allocating zero was never a good idea. it had varying behaviours on varying platforms for as long as i could remember and was one of those special cases to avoid. the allocators didn't necessary even meet the alignment specification either... even if the documents said they did. i remember reading that malloc on windows would return things on 8-byte boundaries only to find pointers ending in 4 (rather than 0 or 8) coming out of it... and as many have pointed out returning null may or may not happen in a number of situations. i've seen it happen enough times when i ask for too much memory to believe that it is a useful fail case to check for...

kazinator 11y ago
realloc(ptr, 0) special case which (on some C libraries only) means the same thing as free(ptr) and not to realloc the object to zero size. That's what the author's resize function expected, which is why his patch is to use free.
- fanf2 11y ago
  C libraries do not conform to the standard if they do not free in this case.
  - masklinn 11y ago
    That requirement was present in C89 and removed in C99, which also added that a 0-sized allocation doesn't have to return a null pointer (whether done by malloc, calloc or realloc)
    - kazinator 11y ago
      Dang, is that right? See, I checked C99 when I posted the comment. I don't know where my old copy of C90 is, but I was mildly surprised. I had that WTF feeling "I could have sworn it was supposed to work that way! In any case, if I'm compiling with -ansi or -std=c90 on a GCC-based platform that is hosted, the library had better behave in the C90 conforming manner. Only if if -std=c99 is used should it take the liberty with realloc(ptr, 0) being like malloc(0).

antirez 11y ago

The fact when you reallocate you mean to trow the data away does not make sense... In general the article seems honestly a bit confused in what it states.

cperciva 11y ago
realloc can change the size of a memory allocation and return the same pointer.
- antirez 11y ago
  What I mean is that, when you say: > Unfortunately, some people are not sane, and have decided that it's equally valid for realloc(p, 0) to return NULL meaning-failure and not free p. What you want is the following semantics: char *my_old_vector = ... ; /* We created / allocated in some way. */ add_important_stuff_to(my_old_vector); char *new_vector = realloc(my_old_vector,size+100); if (new_vector == NULL) { /* I still have the old vector and can recover. */ }

carussell 11y ago

> So what happens if realloc(p, 0) fails? It's going to return NULL, since that's what realloc does if it fails; but should it also free the allocation p? The sane answer is yes If realloc(p, 0) is a synonym for free(p), what does it mean for it to "also free the allocation p" in the event of failure? Unpacking it, the statement seems to be saying the same thing as, "if it fails, then what it should do is to not fail".

cperciva 11y ago
Freeing a memory allocation is guaranteed to never fail. Allocating zero bytes can fail (with dumb C libraries). The question is "if realloc fails to allocate zero bytes, should it still free the original allocation".
- carussell 11y ago
  I'm still not clear on this. What does "allocating zero bytes" mean, if not "free the original allocation"? Keep the malloc metadata structures around but update their size to 0?
  - cperciva 11y ago
    Some systems handle a zero-byte allocation request by allocating a unique non-NULL pointer which cannot be dereferenced. If they run out of unique pointers, they can fail to allocate zero bytes.
- pcwalton 11y ago
  > Allocating zero bytes can fail (with dumb C libraries). They might not be dumb; they might be forced into it to maintain compatibility with buggy programs and/or configure scripts (e.g. [1]). [1]: http://www.openwall.com/lists/musl/2013/01/15/1 http://www.openwall.com/lists/musl/2013/01/15/1
  - cperciva 11y ago
    Unfortunately, as I wrote in the commit message which spawned this thread, this just encourages people to write more buggy programs.

coldpie 11y ago

The REAL problem here is C's garbage way of dealing with errors. NULL is overloaded both to be a valid pointer and an error signal. In the author's ideal world, how would a malloc failure be communicated?

scott_s 11y ago
Null is not a valid pointer, though. It's a valid pointer value, but it's undefined to dereference such a pointer. If the purpose of malloc is to return an address to usable memory, then null it outside of that range. Values outside of the legal range of a function are fine for error conditions. Because of that, null seems like a reasonable return value for error on malloc: I could not allocate memory for you, which I indicate by returning an address that you are not supposed to dereference. However, I do agree it would be better if there was some other kind of error condition, preferably something like the option type from various other languages, or just an easy way to return multiple values like in Go. But C is not that language.
- jzwinck 11y ago
  C can return a struct from a function. That's multiple return values.
  - scott_s 11y ago
    Yeah, but it's cumbersome because you have to define that struct elsewhere, and C does not have elegant mechanisms for destructing in-place.
    - ratboy666 11y ago
      How about #include <stdio.h> typedef struct mystruct { int x; char *p; } mystruct; mystruct func(int n) { return (mystruct) { n, NULL }; } int main(void) { mystruct v = func(1); printf("%d\n", v.x); printf("%d\n", func(2).x); return 0; } Works fine with gcc 5.3.1. Multiple destructuring is (maybe) problematic, but if you really want this.

asveikau 11y ago

This is way too hyperbolic. I don't know what specific "once upon a time" period is being described. However malloc(0), realloc(NULL, x), and realloc(y, 0) has been a portability headache for as long as I have been writing C. None of this is all that surprising.

klodolph 11y ago
The behavior of realloc(NULL, x) is always equivalent to malloc(x) according to the C standard. But yes, I never remember a time when malloc(0) or realloc(y, 0) played nice.
- asveikau 11y ago
  > The behavior of realloc(NULL, x) is always equivalent to malloc(x) according to the C standard. AFAIK if you try to do this with even a fairly recent Microsoft libc it won't work. I haven't checked the standards and history on this one, but keep in mind it's been only a recent development that MS gives a crap about C99. The idea that C89 did not mandate this makes sense to me because I remember now-obsolete Unixes not liking this either. [Edit: The MS documentation claims that it supports realloc(NULL, x) going back quite a while. I know I've been bitten by it not doing that in the current century, however...]
  - masklinn 11y ago
    > The idea that C89 did not mandate this makes sense to me because I remember now-obsolete Unixes not liking this either. C89 did mandate this: > If ptr is a null pointer, the realloc function behaves like the malloc function for the specified size. C89 also mandated that realloc(ptr, 0) be equivalent to free(ptr), which was removed in C99: > If size is zero and ptr is not a null pointer, the object it points to is freed.

acqq 11y ago

What I don't understand is why some programmers invest so much energy in using realloc. When you know that for most cases realloc actually does a new alloc and copy of the existing content, but has the corners of undefined or unwanted behavior, why not doing it in your own function in a way that you fully (for your needs) control?

klodolph 11y ago
You certainly could wrap realloc(), but besides the bizarre/unnecessary behavior with realloc(ptr, 0), it behaves exactly as you would expect: on failure it returns NULL and doesn't touch your pointer, on success it returns the new pointer. It doesn't seem like it requires any more "energy" than using malloc() and programming in C to begin with.
acconsta 11y ago
For large allocations, realloc can remap virtual memory instead of doing a naive copy: http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-and-laziness/ http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-a...
- acqq 11y ago
  That's exactly when I'd use realloc. Looking who wrote the text, I also respect cperciva and believe he must have some good reasons and I'd be glad read which use cases he had, more than just "what's wrong with different reallocs." Because I'm not surprised that the corner cases aren't to everybody's (or even anybody's) satisfaction. It's C. Less is more and all that.
- cperciva 11y ago
  And for small allocations, realloc can expand or shrink an allocation in-place, because it knows where it is placing other memory allocations.
  - acqq 11y ago
    Yes. If the allocation block is 16 bytes for example, the string growing from 10 to 11, 12, 13 etc could still be on the same place. But in which use cases are frequent reallocs actually needed, so much that you can recognize the performance impact? I'd really like to know, as I personally never had such problems. When the single allocations were too expensive I've just used some kind of memory pool. For small stuff realloc is still more expensive than just a few instructions on average when some pool is used.

kazinator 11y ago

The fix is to wrap malloc and friends with your own routines. You can have it so that "my_malloc" always returns NULL on a zero length object. Or, if you prefer, so that it always returns a unique pointer: void *my_malloc(size_t size) { return size ? malloc(size) : 0; /* option A */ } void *my_malloc(size_t size) { return malloc(size ? size : 1); /* option B */ } And you can make my_realloc have the nice freeing behavior: void *my_realloc(void *ptr, size_t size) { return (ptr && size) ? realloc(ptr, size) : !ptr ? my_malloc(size) : (free(ptr), 0); } Note how we still carefully implement a parallel requirement to the one in ISO C. Namely that my_realloc(NULL, size) calls my_malloc(size), regardless of size, just like realloc(NULL, size) is required to be equivalent to malloc(size). We want all the routines in this allocator to be drop-in replacements such that any code which is written to the ISO C allocator spec can be blindly retargetted to use them. Pretty much any serious C program, if indeed it doesn't have an entire allocator of its own, at least wraps the standard one, for the sake of more uniform behaviors, as well as easy retargettability to embedded scenarios. (Not only embedded machines, but say, embedding in a larger application, where you are told "you must use this table of funtion pointers as your allocator" and it doesn't quite look like malloc: zero allocations are not allowed, realloc is missing, ...) In this particular program, realloc is wrapped under a function called resize; resize had to be patched not to rely on realloc(nonnull, 0), but users of resize don't have to change. The programmer doesn't have to look for fifty uses of resize to fix them. However, it's worth it to wrap the functions at a lower level anyway, with an identical API. The C library is far from perfect. Do not expect consistency, completeness, beauty, symmetry and so on. There are worse things in it than realloc(ptr, 0) not behaving in the neat way that you would like. Oh such as: isalpha(str[42]); // str is char array being undefined behavior if char happens to be signed, and str[42] is negative, because isalpha takes an int argument which is expected to hold a positive byte value [0, UCHAR_MAX), and not char value. Whoooops!

klodolph 11y ago
You can make my_realloc() simpler: void *my_realloc(void *ptr, size_t size) { return size ? realloc(ptr, size) : (free(ptr), 0); } According to the C standard, realloc(NULL, size) is guaranteed to be equivalent to malloc(size). From n1548 7.22.3.5 para 3: > If ptr is a null pointer, the realloc function behaves like the malloc function for the specified size. Also, free(0) is well-defined. From n1548 7.22.3.3 para 2: > If ptr is a null pointer, no action occurs.

theseoafs 11y ago

I don't know if the distinction between which behaviors are "sane" and "insane" is a good one here. Evidently the only insane choice is to rebuff the standard in your implementation of the core C memory management functions.