3 ms·

Compiler developers hijacked and twisted the term "Undefined Behavior". Everyone understood what UB was in K&R C - if you write code that the standard doesn't d

Compiler developers hijacked and twisted the term "Undefined Behavior". Everyone understood what UB was in K&R C - if you write code that the standard doesn't define a meaning to, and the compiler outputs what it outputs. If you dereference a null pointer, the compiler outputs a null pointer dereference, and when you hit it at runtime you get the undefined behavior (page fault on modern systems). Nowadays, UB means something completely different - if at any point in time, the compiler reasons out that a piece of code is only reachable via UB, it will assume that this can never happen, and will quietly delete everything downstream: https://godbolt.org/z/EYxWqcfjx https://godbolt.org/z/EYxWqcfjx

torben-friis 8mo ago
Sorry if I’m missing something as this isn’t my field, but shouldn’t the two meanings be roughly equivalent to the user? As in, everything down from UB is only working by an accident of implementation that does not need to hold, and you should explicitly not rely on that. Whether the compiler happens to explicitly make it not ever work or just leaves it to fate should not be relevant.
- anilakar 8mo ago
  My main gripe with UB is that if a compiler is able to detect undefined behavior invocation, it is still allowed to compile (or rather omit) said code instead of crashing.
- Doxin 8mo ago
  No, because the former definition is still something you can rely on given a specific compiler and a specific machine. Hell a bunch of UB was pretty much universal anyway. Compilers would usually still emit sensible code for UB. UB just ment "the spec doesn't define what happens". It didn't use to mean "the compiler can just decide to do any wild thing if your program touches UB anywhere at anytime". Hell, with the modern definition UB can aparantly time travel. you don't even need to execute UB code for it to start doing weird shit in some cases. UB went from "whatever happens when your compiler/hardware runs this is what happens" to "Once a program contains UB the compiler doesn't need to conform to the rest of the spec anymore."
  - torben-friis 8mo ago
    >the former definition is still something you can rely on given a specific compiler and a specific machine. >UB just ment "the spec doesn't define what happens" What comes to mind is that then the written code is operating on a subspec, one that is probably undocumented and maybe even unintended by the specifics of that version and platform. It sounds like it could create a ton of issues, from code that can’t be ported to difficulty in other person grokking the undocumented behavior that is being used. In this regard, as someone that could potentially inherit this code I’d actually want the compiler to stop this potential behavior. Am I missing something? Is the spec not functional enough on its own to rely just on that?
    - xscott 8mo ago
      Very simple code is UB: int handle_untrusted_numbers(int a, int b) { if (a < 0) return ERROR_EXPECTED_NON_NEGATIVE; if (b < 0) return ERROR_EXPECTED_NON_NEGATIVE; int sum = a + b; if (sum < 0) { return ERROR_INTEGER_OVERFLOW; } return do_something_important_with(sum); } Every computer you will ever use has two's complement for signed integers, and the standard recently recognized and codified this fact. However, the UB fanatics (heretics) insisted that not allowing signed overflow is an important opportunity for optimizations, so that last if-statement can be deleted by the compiler and your code quietly doesn't check for overflow any more. There are plenty more examples, but I think this is one of the simplest.
    - Doxin 8mo ago
      I'm not opposed to compilers erroring out on UB. But that's not what happens. Instead of choosing to either proceed and hope all is well, or choosing to stop and error out, compilers instead take the secret third option of breaking your code even more and telling no one.
  - pseudohadamard 8mo ago
    One thing you need to add is that UB can be incredibly subtle and almost impossible to spot even by people with decades of programming experience. However, the compiler - and we're talking almost exclusively gcc here - will spot it and silently break your code. It won't warn "hey, I've spotted UB here!" even with every possible warning enabled, it will just quietly break your code without giving you any indication that it's done so. It's some of the most user-hostile behavior I've ever encountered in an application.
atiedebee 8mo ago
ISO C99 actually defines multiple types of deviating behaviour. What you're describing is closer to implementation-defined behaviour than anything else. The three behaviours relevant in this discussion, from section 3.4: 3.4.1 implementation-defined behavior unspecified behavior where each implementation documents how the choice is made EXAMPLE An example of implementation-defined behavior is the propagation of the high-order bit when a signed integer is shifted right. 3.4.3 undefined behavior behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). An example of undefined behavior is the behavior on integer overflow. 3.4.4 unspecified behavior behavior where this International Standard provides two or more possibilities and imposes no further requirements on which is chosen in any instance An example of unspecified behavior is the order in which the arguments to a function are evaluated. K&R seems to also mention "undefined" and "implementation-defined" behaviour on several occasions. It doesn't specify what is meant by undefined behaviour, but it does indeed seem to be "whatever happens, happens" instead of "you can do whatever you want." But ISO C99 seems to be a lot looser with their definition. Using integer overflow, as in your example, for optimization has been shown to be beneficial by Charles Carruth in a talk he did at CppCon in 2016.[1] I think it would be best to have something similar to Zig's wrapping and saturating addition operators instead, but for that I think it is better to just use Zig (which I personally am very willing to do once they reach 1.0 and other compiler implementations are available).[2] [1] https://youtu.be/yG1OZ69H_-o?si=x-9ALB8JGn5Qdjx_&t=2357 https://youtu.be/yG1OZ69H_-o?si=x-9ALB8JGn5Qdjx_&t=2357 [2] https://ziglang.org/documentation/0.15.2/#Operators https://ziglang.org/documentation/0.15.2/#Operators
- nananana9 8mo ago
  [1] is probably the best counterpoint I've seen, but there are other ways to enable this optimization - the most obvious being to use a register-sized index, which is what's passed to the function anyways. I'd be fine with an intrinsic for this as well (I don't think you'll use it often enough to justify the +%= syntax) It's also worth noting that even with the current very liberal handling of UB, the actual code sample in [1] was still lacking this optimization; so it's not like the liberal UB handling automatically lead to faster code, understanding of the compiler was still needed. The question is one of risk - if the compiler is conservative, you're risking is a slightly more unoptimized code. If the compiler is very liberal and assumes UB never happens, you're risking that it will wipe your overflow check like in my godbolt (I've seen an actual CVEs due to that, although I don't remember the project)

torben-friis 8mo ago

Sorry if I’m missing something as this isn’t my field, but shouldn’t the two meanings be roughly equivalent to the user? As in, everything down from UB is only working by an accident of implementation that does not need to hold, and you should explicitly not rely on that. Whether the compiler happens to explicitly make it not ever work or just leaves it to fate should not be relevant.

anilakar 8mo ago
My main gripe with UB is that if a compiler is able to detect undefined behavior invocation, it is still allowed to compile (or rather omit) said code instead of crashing.
Doxin 8mo ago
No, because the former definition is still something you can rely on given a specific compiler and a specific machine. Hell a bunch of UB was pretty much universal anyway. Compilers would usually still emit sensible code for UB. UB just ment "the spec doesn't define what happens". It didn't use to mean "the compiler can just decide to do any wild thing if your program touches UB anywhere at anytime". Hell, with the modern definition UB can aparantly time travel. you don't even need to execute UB code for it to start doing weird shit in some cases. UB went from "whatever happens when your compiler/hardware runs this is what happens" to "Once a program contains UB the compiler doesn't need to conform to the rest of the spec anymore."
- torben-friis 8mo ago
  >the former definition is still something you can rely on given a specific compiler and a specific machine. >UB just ment "the spec doesn't define what happens" What comes to mind is that then the written code is operating on a subspec, one that is probably undocumented and maybe even unintended by the specifics of that version and platform. It sounds like it could create a ton of issues, from code that can’t be ported to difficulty in other person grokking the undocumented behavior that is being used. In this regard, as someone that could potentially inherit this code I’d actually want the compiler to stop this potential behavior. Am I missing something? Is the spec not functional enough on its own to rely just on that?
  - xscott 8mo ago
    Very simple code is UB: int handle_untrusted_numbers(int a, int b) { if (a < 0) return ERROR_EXPECTED_NON_NEGATIVE; if (b < 0) return ERROR_EXPECTED_NON_NEGATIVE; int sum = a + b; if (sum < 0) { return ERROR_INTEGER_OVERFLOW; } return do_something_important_with(sum); } Every computer you will ever use has two's complement for signed integers, and the standard recently recognized and codified this fact. However, the UB fanatics (heretics) insisted that not allowing signed overflow is an important opportunity for optimizations, so that last if-statement can be deleted by the compiler and your code quietly doesn't check for overflow any more. There are plenty more examples, but I think this is one of the simplest.
  - Doxin 8mo ago
    I'm not opposed to compilers erroring out on UB. But that's not what happens. Instead of choosing to either proceed and hope all is well, or choosing to stop and error out, compilers instead take the secret third option of breaking your code even more and telling no one.
- pseudohadamard 8mo ago
  One thing you need to add is that UB can be incredibly subtle and almost impossible to spot even by people with decades of programming experience. However, the compiler - and we're talking almost exclusively gcc here - will spot it and silently break your code. It won't warn "hey, I've spotted UB here!" even with every possible warning enabled, it will just quietly break your code without giving you any indication that it's done so. It's some of the most user-hostile behavior I've ever encountered in an application.

atiedebee 8mo ago

ISO C99 actually defines multiple types of deviating behaviour. What you're describing is closer to implementation-defined behaviour than anything else. The three behaviours relevant in this discussion, from section 3.4: 3.4.1 implementation-defined behavior unspecified behavior where each implementation documents how the choice is made EXAMPLE An example of implementation-defined behavior is the propagation of the high-order bit when a signed integer is shifted right. 3.4.3 undefined behavior behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). An example of undefined behavior is the behavior on integer overflow. 3.4.4 unspecified behavior behavior where this International Standard provides two or more possibilities and imposes no further requirements on which is chosen in any instance An example of unspecified behavior is the order in which the arguments to a function are evaluated. K&R seems to also mention "undefined" and "implementation-defined" behaviour on several occasions. It doesn't specify what is meant by undefined behaviour, but it does indeed seem to be "whatever happens, happens" instead of "you can do whatever you want." But ISO C99 seems to be a lot looser with their definition. Using integer overflow, as in your example, for optimization has been shown to be beneficial by Charles Carruth in a talk he did at CppCon in 2016.[1] I think it would be best to have something similar to Zig's wrapping and saturating addition operators instead, but for that I think it is better to just use Zig (which I personally am very willing to do once they reach 1.0 and other compiler implementations are available).[2] [1] https://youtu.be/yG1OZ69H_-o?si=x-9ALB8JGn5Qdjx_&t=2357 https://youtu.be/yG1OZ69H_-o?si=x-9ALB8JGn5Qdjx_&t=2357 [2] https://ziglang.org/documentation/0.15.2/#Operators https://ziglang.org/documentation/0.15.2/#Operators

nananana9 8mo ago
[1] is probably the best counterpoint I've seen, but there are other ways to enable this optimization - the most obvious being to use a register-sized index, which is what's passed to the function anyways. I'd be fine with an intrinsic for this as well (I don't think you'll use it often enough to justify the +%= syntax) It's also worth noting that even with the current very liberal handling of UB, the actual code sample in [1] was still lacking this optimization; so it's not like the liberal UB handling automatically lead to faster code, understanding of the compiler was still needed. The question is one of risk - if the compiler is conservative, you're risking is a slightly more unoptimized code. If the compiler is very liberal and assumes UB never happens, you're risking that it will wipe your overflow check like in my godbolt (I've seen an actual CVEs due to that, although I don't remember the project)