6 ms·

From what I understand: 1. Initially, they just wanted to give compiler makers more freedom: both in the sense "do whatever is simplest" and "do something plat

From what I understand: 1. Initially, they just wanted to give compiler makers more freedom: both in the sense "do whatever is simplest" and "do something platform-specific which dev wants". 2. Compiler devs found that they can use UB for optimization: e.g. if we assume that a branch with UB is unreachable we can generate more efficient code. 3. Sadly, compiler devs started to exploit every opportunity for optimization, e.g. removing code with a potential segfault. I.e. people who made a standard thought that compiler would remove no-op call to memcpy, but GCC removes the whole branch which makes the call as it considers the whole branch impossible. Standard makers thought that compiler devs would be more reasonable

kllrnohj 2y ago
> Standard makers thought that compiler devs would be more reasonable This is a bit of a terrible take? Compiler devs never did anything "unreasonable", they didn't sit down and go "mwahahaha we can exploit the heck out of UB to break everything!!!!" Rather, repeatedly applying a series of targeted optimizations, each one in isolation being "reasonable", results in an eventual "unreasonable" total transformation. But this is more an emergent property of modern compilers having hundreds of optimization passes. At the time the standards were created, the idea of compilers applying so many optimization passes was just not conceivable. Compilers struggled to just do basic compilation. The assumption was a near 1:1 mapping between code & assembly, and that just didn't age well at all.
- LegionMammal978 2y ago
  One could argue that "optimizing based on signed overflow" was an unreasonable step to take, since any given platform will have some sane, consistent behavior when the underlying instructions cause an overflow. A developer using signed operations without poring over the standard might have easily expected incorrect values (or maybe a trap if the platform likes to use those), but not big changes in control flow. In my experience, signed overflow is generally the biggest cause of "they're putting UB in my reasonable C code!", followed by the rules against type punning, which are violated every day by ordinary usage of the POSIX socket functions.
  - kllrnohj 2y ago
    > One could argue that "optimizing based on signed overflow" was an unreasonable step to take That optimization allows using 64-bit registers / offset loads for signed ints which it can't do if it has to overflow, since that overflow must happen at 32-bits. That's not an uncommon thing.
  - uecker 2y ago
    I started to like signed overflow rules, because it is really easy to find problems using sanitizers. The strict aliasing rules are not violated by typical POSIX socket code as a cast to a different pointer type, i.e. `struct sockaddr` by itself is well-defined behavior. (and POSIX could of course just define something even if ISO C leaves it undefined, but I don't think this is needed here)
    - LegionMammal978 2y ago
      > The strict aliasing rules are not violated by typical POSIX socket code as a cast to a different pointer type, i.e. `struct sockaddr` by itself is well-defined behavior. Basically all usage of sendmsg() and recvmsg() with a static char[N] buffer is UB, is one big example I've run into. Unless you memcpy every value into and out of the buffer, which literally no one does. Also, reading sa_family from the output of accept() (or putting it into a struct sockaddr_storage and reading ss_family) is UB, unless you memcpy it out, which literally no one does.
      - uecker 2y ago
        Using a static char buffer would indeed UB but we just made the change to C2Y that this ok (and in practice it always was). Incorrect use of sockaddr_storage may lead to UB. But again, most socket code I see is actually correct.
- lmm 2y ago
  > Compiler devs never did anything "unreasonable", they didn't sit down and go "mwahahaha we can exploit the heck out of UB to break everything!!!!" Many compiler devs are on record gleefully responding to bug reports with statements on the lines of "your code has undefined behaviour according to the standard, we can do what we like with it, if you don't like it write better code". Less so in recent years as they've realised this was a bad idea or at least a bad look, but in the '00s it was a normal part of the culture.
- killerstorm 2y ago
  What stops compiler makers from treating UB as platform-specific behavior rather than as something which cannot happen? "You are not allowed to do this, and thus..." reasoning assumes that programmers are language lawyers, which is unreasonable.
  - kllrnohj 2y ago
    bool foo(some_struct* bar) { if (bar->blah()) { return true; } if (bar == nullptr) { return false; } return true; } Can the compiler eliminate that nullptr comparison in your opinion yes or no? While this example looks stupid, after inlining it's quite plausible to end up with code in this type of a pattern. Dereferencing a nullptr is UB, and typically the "platform-specific" behavior is a crash, so... why should that if statement remain? And then if it can't remain, why should an explicit `_Nonnull` assertion have different behavior than an explicit deref? What if the compiler can also independently prove that some_struct->blah() always evaluates to false, so it eliminates that entire branch - does the `if (bar == nullptr)` still need to remain in that specific case? If so, why? The code was the same in both cases, the compiler just got better at eliminating dead code.
UncleMeat 2y ago
There isn't a "find UB branches" pass that is seeking out this stuff. Instead what happens is that you have something like a constant folding or value constraint pass that computes a set of possible values that a variable can hold at various program points by applying constraints of various options. Then you have a dead code elimination pass that identifies dead branches. This pass doesn't know why the "dest" variable can't hold the NULL value at the branch. It just knows that it can't, so it kills the branch. Imagine the following code: int x = abs(get_int()); if (x < 0) { // do stuff } Can the compiler eliminate the branch? Of course. All that's happened here is that the constraint propagation feels "reasonable" to you in this case and "unreasonable" to you in the memcpy case.
- meonukk 2y ago
  Why is it allowed to eliminate the branch? In most architectures abs(INT_MIN) returns INT_MIN which is negative
  - plorkyeran 2y ago
    Calling abs(INT_MIN) on twos-complement machine is not allowed by the C standard. The behavior of abs() is undefined if the result would not fit in the return value.
    - ryao 2y ago
      Where does it say that? I thought this was a famous example from formal methods showing why something really simple could be wrong. It would be strange for the standard to say to ignore it. The behavior is also well defined in two’s complement. People just don’t like it.
      - plorkyeran 2y ago
        https://busybox.net/~landley/c99-draft.html#7.20.6.1 https://busybox.net/~landley/c99-draft.html#7.20.6.1 "The abs, labs, and llabs functions compute the absolute value of an integer j. If the result cannot be represented, the behavior is undefined. (242)" 242 The absolute value of the most negative number cannot be represented in two's complement.
mjevans 2y ago
More reasonable: Emit a warning or error to make the code and human writing it better. NOT-reasonable: silently 'optimize' a 'gotcha' into behavior the programmer(s) didn't intend.
- gpderetta 2y ago
  NOT-reasonable: expecting the compiler to read the programmer's mind.
  - mjevans 2y ago
    OK, you want a FORMAL version? Acceptable UB: Do the exact same type of operation as for defined behavior, even if the result is defined by how the underlying hardware works. NOT-acceptable UB: Perform some operation OTHER than the same as if it were the valid code path, EXCEPT: Failure to compile or a warning message stating which code has been transformed into what other operation as a result of UB.
    - gpderetta 2y ago
      I don't understand, if the operation is not defined, what exactly the compiler should do? If I tell you "open the door", that implies that the door is there. If the door is not there, how would you still open the door? Concretely, what do you expect this to return: #include <cstddef> void sink(ptrdiff_t); ptrdiff_t source(); int foo() { int x = 1; int y; sink(&y-&x); *(&y - source()) = 42; return x; } assuming that source() returns the parameter passed to sink()? Incidentally I had to launder the offset through sink/source, because GCC has a must-alias oracle to mitigate miscompiling some UB code, so in a way it already caters to you.
      - mjevans 2y ago
        Evaluated step by step... Offhand, *sink(&y-&x);* the compiler is not _required_ to lay out variables adjacently. So the computation of the pointers fed to sink does not have to be defined and might not be portable. It would be permissible for the compiler to refuse to compile that ('line blah, op blah' does not conform the the standard's allowed range of behavior). It would also be permissible to just allow that operation to happen. It's the difference of two pointer sized units being passed. That's the operation the programmer wrote, that's the operation that will happen. Do not verify bounds or alter behavior because the compiler could calculate that the value happens to be PTRMAX-sizeof(int)+1 (it placed X and Y in reverse of how a naive assumption might assume). The = 42 line might write to any random address in memory. Again, just compile the code to perform the operation. If that happens to write 42 somewhere in the stack frame that leads to the program corrupting / a segfault that's fine. If the compiler says 'wait that's not a known memory location' or 'that's going to write onto the protected stack!' it can ALSO refuse to compile and say why that code is not valid. I would expect valid results to be a return of: 42, 1 (possibly with a warning message about undefined operations and the affected lines), OR the program does not compile and there is an error message which says what's wrong.

kllrnohj 2y ago

> Standard makers thought that compiler devs would be more reasonable This is a bit of a terrible take? Compiler devs never did anything "unreasonable", they didn't sit down and go "mwahahaha we can exploit the heck out of UB to break everything!!!!" Rather, repeatedly applying a series of targeted optimizations, each one in isolation being "reasonable", results in an eventual "unreasonable" total transformation. But this is more an emergent property of modern compilers having hundreds of optimization passes. At the time the standards were created, the idea of compilers applying so many optimization passes was just not conceivable. Compilers struggled to just do basic compilation. The assumption was a near 1:1 mapping between code & assembly, and that just didn't age well at all.

LegionMammal978 2y ago
One could argue that "optimizing based on signed overflow" was an unreasonable step to take, since any given platform will have some sane, consistent behavior when the underlying instructions cause an overflow. A developer using signed operations without poring over the standard might have easily expected incorrect values (or maybe a trap if the platform likes to use those), but not big changes in control flow. In my experience, signed overflow is generally the biggest cause of "they're putting UB in my reasonable C code!", followed by the rules against type punning, which are violated every day by ordinary usage of the POSIX socket functions.
- kllrnohj 2y ago
  > One could argue that "optimizing based on signed overflow" was an unreasonable step to take That optimization allows using 64-bit registers / offset loads for signed ints which it can't do if it has to overflow, since that overflow must happen at 32-bits. That's not an uncommon thing.
- uecker 2y ago
  I started to like signed overflow rules, because it is really easy to find problems using sanitizers. The strict aliasing rules are not violated by typical POSIX socket code as a cast to a different pointer type, i.e. `struct sockaddr` by itself is well-defined behavior. (and POSIX could of course just define something even if ISO C leaves it undefined, but I don't think this is needed here)
  - LegionMammal978 2y ago
    > The strict aliasing rules are not violated by typical POSIX socket code as a cast to a different pointer type, i.e. `struct sockaddr` by itself is well-defined behavior. Basically all usage of sendmsg() and recvmsg() with a static char[N] buffer is UB, is one big example I've run into. Unless you memcpy every value into and out of the buffer, which literally no one does. Also, reading sa_family from the output of accept() (or putting it into a struct sockaddr_storage and reading ss_family) is UB, unless you memcpy it out, which literally no one does.
    - uecker 2y ago
      Using a static char buffer would indeed UB but we just made the change to C2Y that this ok (and in practice it always was). Incorrect use of sockaddr_storage may lead to UB. But again, most socket code I see is actually correct.
lmm 2y ago
> Compiler devs never did anything "unreasonable", they didn't sit down and go "mwahahaha we can exploit the heck out of UB to break everything!!!!" Many compiler devs are on record gleefully responding to bug reports with statements on the lines of "your code has undefined behaviour according to the standard, we can do what we like with it, if you don't like it write better code". Less so in recent years as they've realised this was a bad idea or at least a bad look, but in the '00s it was a normal part of the culture.
killerstorm 2y ago
What stops compiler makers from treating UB as platform-specific behavior rather than as something which cannot happen? "You are not allowed to do this, and thus..." reasoning assumes that programmers are language lawyers, which is unreasonable.
- kllrnohj 2y ago
  bool foo(some_struct* bar) { if (bar->blah()) { return true; } if (bar == nullptr) { return false; } return true; } Can the compiler eliminate that nullptr comparison in your opinion yes or no? While this example looks stupid, after inlining it's quite plausible to end up with code in this type of a pattern. Dereferencing a nullptr is UB, and typically the "platform-specific" behavior is a crash, so... why should that if statement remain? And then if it can't remain, why should an explicit `_Nonnull` assertion have different behavior than an explicit deref? What if the compiler can also independently prove that some_struct->blah() always evaluates to false, so it eliminates that entire branch - does the `if (bar == nullptr)` still need to remain in that specific case? If so, why? The code was the same in both cases, the compiler just got better at eliminating dead code.

UncleMeat 2y ago

There isn't a "find UB branches" pass that is seeking out this stuff. Instead what happens is that you have something like a constant folding or value constraint pass that computes a set of possible values that a variable can hold at various program points by applying constraints of various options. Then you have a dead code elimination pass that identifies dead branches. This pass doesn't know why the "dest" variable can't hold the NULL value at the branch. It just knows that it can't, so it kills the branch. Imagine the following code: int x = abs(get_int()); if (x < 0) { // do stuff } Can the compiler eliminate the branch? Of course. All that's happened here is that the constraint propagation feels "reasonable" to you in this case and "unreasonable" to you in the memcpy case.

meonukk 2y ago
Why is it allowed to eliminate the branch? In most architectures abs(INT_MIN) returns INT_MIN which is negative
- plorkyeran 2y ago
  Calling abs(INT_MIN) on twos-complement machine is not allowed by the C standard. The behavior of abs() is undefined if the result would not fit in the return value.
  - ryao 2y ago
    Where does it say that? I thought this was a famous example from formal methods showing why something really simple could be wrong. It would be strange for the standard to say to ignore it. The behavior is also well defined in two’s complement. People just don’t like it.
    - plorkyeran 2y ago
      https://busybox.net/~landley/c99-draft.html#7.20.6.1 https://busybox.net/~landley/c99-draft.html#7.20.6.1 "The abs, labs, and llabs functions compute the absolute value of an integer j. If the result cannot be represented, the behavior is undefined. (242)" 242 The absolute value of the most negative number cannot be represented in two's complement.

mjevans 2y ago

More reasonable: Emit a warning or error to make the code and human writing it better. NOT-reasonable: silently 'optimize' a 'gotcha' into behavior the programmer(s) didn't intend.

gpderetta 2y ago
NOT-reasonable: expecting the compiler to read the programmer's mind.
- mjevans 2y ago
  OK, you want a FORMAL version? Acceptable UB: Do the exact same type of operation as for defined behavior, even if the result is defined by how the underlying hardware works. NOT-acceptable UB: Perform some operation OTHER than the same as if it were the valid code path, EXCEPT: Failure to compile or a warning message stating which code has been transformed into what other operation as a result of UB.
  - gpderetta 2y ago
    I don't understand, if the operation is not defined, what exactly the compiler should do? If I tell you "open the door", that implies that the door is there. If the door is not there, how would you still open the door? Concretely, what do you expect this to return: #include <cstddef> void sink(ptrdiff_t); ptrdiff_t source(); int foo() { int x = 1; int y; sink(&y-&x); *(&y - source()) = 42; return x; } assuming that source() returns the parameter passed to sink()? Incidentally I had to launder the offset through sink/source, because GCC has a must-alias oracle to mitigate miscompiling some UB code, so in a way it already caters to you.
    - mjevans 2y ago
      Evaluated step by step... Offhand, *sink(&y-&x);* the compiler is not _required_ to lay out variables adjacently. So the computation of the pointers fed to sink does not have to be defined and might not be portable. It would be permissible for the compiler to refuse to compile that ('line blah, op blah' does not conform the the standard's allowed range of behavior). It would also be permissible to just allow that operation to happen. It's the difference of two pointer sized units being passed. That's the operation the programmer wrote, that's the operation that will happen. Do not verify bounds or alter behavior because the compiler could calculate that the value happens to be PTRMAX-sizeof(int)+1 (it placed X and Y in reverse of how a naive assumption might assume). The = 42 line might write to any random address in memory. Again, just compile the code to perform the operation. If that happens to write 42 somewhere in the stack frame that leads to the program corrupting / a segfault that's fine. If the compiler says 'wait that's not a known memory location' or 'that's going to write onto the protected stack!' it can ALSO refuse to compile and say why that code is not valid. I would expect valid results to be a return of: 42, 1 (possibly with a warning message about undefined operations and the affected lines), OR the program does not compile and there is an error message which says what's wrong.