3 ms·

One thing that confuses me about the C committee is their inconsistent attitude towards breaking existing code. For example, in C89-C17, calling realloc with a

by ndiddy 3y ago

One thing that confuses me about the C committee is their inconsistent attitude towards breaking existing code. For example, in C89-C17, calling realloc with a size of 0 would free the referenced buffer and return a NULL pointer. This helped use cases where realloc was called in a loop on data that changes size. In C23, they changed it so calling realloc with a size of 0 was undefined behavior. This breaks any code that was doing what was previously idiomatic realloc behavior. It's also hard to find in existing codebases because the length passed to realloc will be a variable in most cases. However, the C committee refused to define NULL as (void *)0 because they were afraid of breaking existing library implementations that define NULL as 0. They instead introduced a new nullptr keyword that's defined as (void *)0. I just don't understand the inconsistency.

veltas 3y ago
https://sourceware.org/bugzilla/show_bug.cgi?id=12547 https://sourceware.org/bugzilla/show_bug.cgi?id=12547 contains some info on where this discussion started, basically there is ambiguity in the old interface and rather than trying to rule on a sensible interpretation the standards committee agreed to make the behaviour obscelescent, and then later undefined, as even now at least one BSD seems to refuse to free memory with a size argument of zero. This was a defect in BSD and the POSIX specification (which literally says it's not intending to contradict the C standard anywhere at the top of every function document), but the committee somehow managed to believe this was cause to deprecate behaviour of realloc() with size zero for 99% of the world. The defect report and associated documents don't seem to be very explicit about the potential impact of the choice to make the behaviour obscelescent in the first place, which might be why this compatibility break was accepted (and eventually completed by making it all undefined). https://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400 https://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm... The approach is inconsistent because the committee isn't perfect and every single change to the standard, especially defects, must attempt to best preserve compatibility on a nuanced case-by-case basis, and sometimes when you're in the committee I suspect it's harder to see the wood for the trees. realloc() was broken trying to address a defect report that suggested C's lax definition was potentially a cause of double-frees, a security nightmare, but probably exaggerated in hindsight; and not really made that explicit, although I haven't checked all the minutes and attended the meetings so maybe there was a more conscious decision to deprecate the free-on-zero rule.
- aw1621107 3y ago
  There's a little more reasoning in the associated C2x proposal N2464 [0]: > [DR 400] was resolved by loosening the requirements in the standard to allow for the existing range of implementations and included in C17. > N2438 Clarification Request requested further clarification [...] > Discussion at the Ithaca meeting and on the mailing list suggested that a call to realloc with a size of 0 be classified as undefined behavior. > [snip] > Classifying a call to realloc with a size of 0 as undefined behavior would allow POSIX to define the otherwise undefined behavior however they please. [0]: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf
  - veltas 3y ago
    Yes but the well was poisoned by marking it for removal / obscelescent in the first place. Making it undefined behaviour is the next logical step after that.
quelsolaar 3y ago
Making realloc with a size 0 be undefined is a way of maintaining existing code, because it means that different implementations kan keep doing what they are doing. Some implementations free the pointer, some create a zero size allocation. Since there are code depending on both behaviours, we cant define a consistent behaviour, without breaking code that depends on another behaviour. This way we acknowledge the issue without breaking anything. There was a discussion if it should be defined as "platform defined" instead, but the wg14 wanted to indicate that they discourage this use of realloc, and undefined behaviour was a way of doing this.
- ndiddy 3y ago
  Making realloc(buff, 0) undefined means that the compiler is permitted to do nothing instead of calling free on buff. Because this both complies with the C23 standard and makes the computer do less work, I won't be surprised if we start seeing compilers applying this optimization. Hopefully we'll at least get a "-frealloc-size-zero-free" compiler flag or something. BTW, if the "we" implies you're a WG14 member, I wanted to thank you for helping move the language forward. C23 has tons of nice features that makes it compelling to move to (#embed, true/false being keywords, binary literals, built-in endianness macros, etc), which is why I'm concerned about adding more undefined behavior to the standard because some BSD variants didn't comply with the previous standard.
  - quelsolaar 3y ago
    I am in the wg14. But you should not thank me for those features. I'm not very happy with C23. We are working on making UB more understood, and to try to eliminate some UB. To be clear: BSD was not breaking the standard, the standard didn't spell out what the correct action was. So different people had different interpretations. The standard states that things that aren't explicitly defined are UB, so one interpretation is that it was UB to begin with.
- deleted 3y ago
  [deleted]
- matheusmoreira 3y ago
chrsw 3y ago
Maybe they care more about breaking library code than user code? Not saying I agree but maybe that explains some of their decisions. I don't know if changing the realloc() NULL behavior to undefined necessarily breaks code. It definitely leaves the door open to the possibility of breaking code on standard library users but it's still up to the implementation to change the behavior for the code to break.
JonChesterfield 3y ago
Oh great. I've got realloc(0) in some code. Here's hoping clang and gcc ignore that.
- steveklabnik 3y ago
  https://godbolt.org/z/esssnq7fs https://godbolt.org/z/esssnq7fs Looks like gcc does not take advantage of this, but clang does.

by ndiddy 3y ago

veltas 3y ago

https://sourceware.org/bugzilla/show_bug.cgi?id=12547 https://sourceware.org/bugzilla/show_bug.cgi?id=12547 contains some info on where this discussion started, basically there is ambiguity in the old interface and rather than trying to rule on a sensible interpretation the standards committee agreed to make the behaviour obscelescent, and then later undefined, as even now at least one BSD seems to refuse to free memory with a size argument of zero. This was a defect in BSD and the POSIX specification (which literally says it's not intending to contradict the C standard anywhere at the top of every function document), but the committee somehow managed to believe this was cause to deprecate behaviour of realloc() with size zero for 99% of the world. The defect report and associated documents don't seem to be very explicit about the potential impact of the choice to make the behaviour obscelescent in the first place, which might be why this compatibility break was accepted (and eventually completed by making it all undefined). https://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_400 https://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm... The approach is inconsistent because the committee isn't perfect and every single change to the standard, especially defects, must attempt to best preserve compatibility on a nuanced case-by-case basis, and sometimes when you're in the committee I suspect it's harder to see the wood for the trees. realloc() was broken trying to address a defect report that suggested C's lax definition was potentially a cause of double-frees, a security nightmare, but probably exaggerated in hindsight; and not really made that explicit, although I haven't checked all the minutes and attended the meetings so maybe there was a more conscious decision to deprecate the free-on-zero rule.

aw1621107 3y ago
There's a little more reasoning in the associated C2x proposal N2464 [0]: > [DR 400] was resolved by loosening the requirements in the standard to allow for the existing range of implementations and included in C17. > N2438 Clarification Request requested further clarification [...] > Discussion at the Ithaca meeting and on the mailing list suggested that a call to realloc with a size of 0 be classified as undefined behavior. > [snip] > Classifying a call to realloc with a size of 0 as undefined behavior would allow POSIX to define the otherwise undefined behavior however they please. [0]: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2464.pdf
- veltas 3y ago
  Yes but the well was poisoned by marking it for removal / obscelescent in the first place. Making it undefined behaviour is the next logical step after that.

quelsolaar 3y ago

Making realloc with a size 0 be undefined is a way of maintaining existing code, because it means that different implementations kan keep doing what they are doing. Some implementations free the pointer, some create a zero size allocation. Since there are code depending on both behaviours, we cant define a consistent behaviour, without breaking code that depends on another behaviour. This way we acknowledge the issue without breaking anything. There was a discussion if it should be defined as "platform defined" instead, but the wg14 wanted to indicate that they discourage this use of realloc, and undefined behaviour was a way of doing this.

ndiddy 3y ago
Making realloc(buff, 0) undefined means that the compiler is permitted to do nothing instead of calling free on buff. Because this both complies with the C23 standard and makes the computer do less work, I won't be surprised if we start seeing compilers applying this optimization. Hopefully we'll at least get a "-frealloc-size-zero-free" compiler flag or something. BTW, if the "we" implies you're a WG14 member, I wanted to thank you for helping move the language forward. C23 has tons of nice features that makes it compelling to move to (#embed, true/false being keywords, binary literals, built-in endianness macros, etc), which is why I'm concerned about adding more undefined behavior to the standard because some BSD variants didn't comply with the previous standard.
- quelsolaar 3y ago
  I am in the wg14. But you should not thank me for those features. I'm not very happy with C23. We are working on making UB more understood, and to try to eliminate some UB. To be clear: BSD was not breaking the standard, the standard didn't spell out what the correct action was. So different people had different interpretations. The standard states that things that aren't explicitly defined are UB, so one interpretation is that it was UB to begin with.
deleted 3y ago
[deleted]
matheusmoreira 3y ago

chrsw 3y ago

Maybe they care more about breaking library code than user code? Not saying I agree but maybe that explains some of their decisions. I don't know if changing the realloc() NULL behavior to undefined necessarily breaks code. It definitely leaves the door open to the possibility of breaking code on standard library users but it's still up to the implementation to change the behavior for the code to break.

JonChesterfield 3y ago

Oh great. I've got realloc(0) in some code. Here's hoping clang and gcc ignore that.

steveklabnik 3y ago
https://godbolt.org/z/esssnq7fs https://godbolt.org/z/esssnq7fs Looks like gcc does not take advantage of this, but clang does.