9 ms·

But that's the thing: dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens; a C program execu

But that's the thing: dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens; a C program executing undefined behaviour is _invalid C_. Thus, any time you dereference a pointer you also implicitly proimise the compiler that this pointer will _never_ be an invalid pointer. Same with signed arithmetic: you are telling the compiler that your arithmetic is guaranteed to never overflow. Whether this is a good or bad thing is of course a legitimate (and good!) question, but for writing C today that's how the language is specced and a reality the programmer needs to take care to avoid, just like a bunch of other things that C leaves to the programmer like remembering to clean up allocated resources when they're no longer needed.

ClumsyPilot 4y ago
< you are telling the compiler that your arithmetic is guaranteed to never overflow. Any time your are dealing with data from real, physical sensors or third-party APIs, this is an impossible guarantee to give - they could literslly break.
- compiler-guy 4y ago
  of course it is possible. One must validate the input before performing arithmetic on signed integers.
  - deleted 4y ago
    [deleted]
  - imtringued 4y ago
    The compiler is allowed to assume that the data will never overflow so it is also allowed to get rid of overflow checks. Now imagine writing a complex validation routine that if it were violated would result in undefined behavior for every value that the validation rejects, a sufficiently smart compiler is allowed to simply remove your validation code, leaving you with no validation whatsoever.
    - deleted 4y ago
      [deleted]
    - AlotOfReading 4y ago
      It can get rid of checks that test whether the results of a previous operation have overflowed. It can't eliminate checks that test whether a subsequent operation will overflow and abort because that would be changing semantics. C is an absolute minefield of undefined behavior, but let's be accurate about the things it does wrong.
    - comex 4y ago
      There are several ways to do overflow checks safely (i.e. without undefined behavior), though the ergonomics are not always ideal. C23 somewhat improves the situation with <stdckdint.h>, a standardized version of GCC’s __builtin_add_overflow and friends. That has ergonomics issues too due to its verbosity, but at least it’s hard to screw up.
    - dllthomas 4y ago
      It can't (correctly) remove checks that would have prevented the undefined behavior, because that is changing the semantics of a program that (as written) does not trigger undefined behavior.
    - kaba0 4y ago
      That doesn’t sound like sound logic to me. The compiler assumes that at the point you have the arithmetics, they won’t overflow. This hinges upon all the former state of the program. If it can prove that the given integers won’t overflow (e.g. due to a previous, redundant check) then it can indeed remove a conditional, but the compiler can’t change the observable behavior of the program.
    - jefftk 4y ago
      The compiler is allowed to assume that if x and y are signed ints: if (x < 0 || y < 0) return -1; return x * y; Will not overflow. And if you try to check for overflow with: if (x < 0 || y < 0) return -1; if (x * y < 0) return -2; return x*y; Then yes, the compiler is within spec to remove your check because the only situation in which you could hit that check would be after signed integer overflow, which it is allowed to assume won't happen. One way to implement this check in GCC where the compiler will respect it would be: if (x < 0 || y < 0) return -1; int z; if (__builtin_smul_overflow(x, y, &z)) return -2; return z;
jenadine 4y ago
> dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens; Sure, but it didn't have to be like this. They could have said it is unspecified wihout allowing the compiler to assume it doesn't happen. Would C have been better if the spec was different?
- wbl 4y ago
  That's not what's happening. Because it's unspecified transforms that are safe in the absence of that behavior are safe to apply since they preserve the semantics. Even something like register allocation requires knowledge of what pointers point to.
marssaxman 4y ago
> But that's the thing: dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens It would be more helpful for the compiler to assume that it does not know what will happen. This is how C actually worked for many years. > a C program executing undefined behaviour is _invalid C_. That was not formerly the case, and it is not always helpful to redefine C in this way. Sometimes you really are not trying to write portable code, and you really do want the behavior you know that the target machine will give you, even if the C spec doesn't require it.
- account42 4y ago
  That's OK, you can compile your program with -O0 if that's the behavior you want from your compiler.
  - butlerm 4y ago
    There are many optimizations that a compiler can perform without relying on the optimization level to determine how to pervert your program that day. If different optimization levels produce different results that is bad thing, something to be avoided, not encouraged. If it is really necessary to generate random code when some anomalous situation is encountered, that should be a special option to enable dangerous non-deterministic if-you-made-a-mistake-we-will-delete-parts-of-your-program type behavior. I wouldn't consider that optimization though, more like disabling all your compiler's safety features.
    - gpderetta 4y ago
      Which optimizations?
      - butlerm 4y ago
        Loop unrolling for loops that have a static or range bounded number of iterations is a good example. Others include constant expression evaluation, dead code elimination, common subexpression elimination, and static function inlining.
somat 4y ago
Always missing in this argument is the logic of how to go from "undefined" to "can never happen". If the spec did not want it to happen they would have said "can not happen/illegal". but no, it is undefined by the spec. The spec knows that it can and will happen they just did not want to pin down the behavior of the compiler. So the compiler optimization team saying "we can assume this will never happen" is a blind almost maliciously complaint viewpoint.
- aw1621107 4y ago
  > Always missing in this argument is the logic of how to go from "undefined" to "can never happen". The reasoning is something like "if we assume UB doesn't happen, but it does happen, the resulting behavior is unpredictable. This is allowed by the standard, though, because UB allows for any behavior, including that produced by assuming UB doesn't happen." In other words, major implementations treat UB as preconditions. Violating those preconditions gets you Interesting Results (TM), but that's allowed by the standard because "unpredictable results " really means unpredictable results. For example, null pointer dereference is UB. If an implementation assumes null pointers can never be dereferenced, it can better optimize some code. If it turns out a null pointer is dereferenced, the argument is that whatever happens then is still permitted by the standard as the standard does not define any program semantics for programs containing UB.
- uecker 4y ago
  I agree that this does not follow from the wording in the standard and I am relatively sure that this was originally not implied. But this view point is repeated quite often nowadays. I think this is because prominent compiler developers promoted this point of view and used this for blaming the user ("Because you have UB in your program it is completely invalid. It is now ok that the compiler breaks it, and it is alone your fault".) The other response to your post is correct so, but this explanation would not allow UB to affect prior observable behavior.
- gpderetta 4y ago
  Ex falso quodlibet
mpweiher 4y ago
> is undefined behaviour, which means the compiler is allowed to assume it never happens No it's not. Or let me rephrase that. The standard says the following: Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). Which of these is "the compiler is allowed to assume it does not happen"?
- User23 4y ago
  > Which of these is "the compiler is allowed to assume it does not happen"? I agree with where you're coming from, but "ignoring the situation completely with unpredictable results" sounds like it pretty much fits the bill. Pretending something doesn't happen sounds a lot like ignoring it completely to me. What the standard obviously does exclude is nonsense like intentionally reformatting your drives.
  - mpweiher 4y ago
    > Pretending something doesn't happen sounds a lot like ignoring it completely to me. Quite the opposite. Assuming it doesn't happen (not "pretending"), is very much not ignoring the situation, at least if you then act on that assumption that it does not happen. Ignoring it just lets it happen when it does, so if the program specifies an out-of-bounds access, the compiler generates code for an out-of-bounds access, ignoring the fact that it is an out of bounds access.
    - aw1621107 4y ago
      > Assuming it doesn't happen (not "pretending"), is very much not ignoring the situation, at least if you then act on that assumption that it does not happen. I'm not sure how assuming UB doesn't happen is distinguishable from a choosing to ignore the situation every time one comes up. You get the same result either way. For example, a compiler can assume that null pointers are never dereferenced, or every time a null pointer is/may be dereferenced it can just "ignore the situation" with the dereference. I'm not seeing a functional difference here. This is, of course, subject to the minor problem that "situation" is arguably underspecified. Compiler writers appear to interpret it as something akin to "code path" (so "ignoring the situation" means "ignoring code paths invoking UB"), while UB-goes-too-far proponents appear to interpret it more broadly, more like "the fact that UB can/will happen" (so "ignoring the situation" means "ignore the fact UB will/may happen"). > Ignoring it just lets it happen when it does, so if the program specifies an out-of-bounds access, the compiler generates code for an out-of-bounds access, ignoring the fact that it is an out of bounds access. Why wouldn't this fall under "behaving during translation or program execution in a documented manner characteristic of the environment" instead?
pygy_ 4y ago
> [whatever] is undefined behaviour, which means the compiler is allowed to assume it never happens; That interpretation is the root of the problem. Compilers authors use it to implement outright user hostile behavior in the name of elusive performance. Why would you spend resources looking for zero days when you can have a few LLVM contributors plant them in every program "as an optimization"?
- Narishma 4y ago
  So why not use a different compiler?
  - pygy_ 4y ago
    Would you write it? You can use the same compiler with a language whose design committee isn't deliberately user hostile, like Rust (where UB-like behaviors in safe code are considered soundness bugs). https://runrust.miraheze.org/wiki/Undefined_Behavior https://runrust.miraheze.org/wiki/Undefined_Behavior
    - saagarjha 4y ago
      Rust makes guarantees for safe code that undefined behavior would violate. C(++) has no such mode and as such a comparison cannot be made.

ClumsyPilot 4y ago

< you are telling the compiler that your arithmetic is guaranteed to never overflow. Any time your are dealing with data from real, physical sensors or third-party APIs, this is an impossible guarantee to give - they could literslly break.

compiler-guy 4y ago
of course it is possible. One must validate the input before performing arithmetic on signed integers.
- deleted 4y ago
  [deleted]
- imtringued 4y ago
  The compiler is allowed to assume that the data will never overflow so it is also allowed to get rid of overflow checks. Now imagine writing a complex validation routine that if it were violated would result in undefined behavior for every value that the validation rejects, a sufficiently smart compiler is allowed to simply remove your validation code, leaving you with no validation whatsoever.
  - deleted 4y ago
    [deleted]
  - AlotOfReading 4y ago
    It can get rid of checks that test whether the results of a previous operation have overflowed. It can't eliminate checks that test whether a subsequent operation will overflow and abort because that would be changing semantics. C is an absolute minefield of undefined behavior, but let's be accurate about the things it does wrong.
  - comex 4y ago
    There are several ways to do overflow checks safely (i.e. without undefined behavior), though the ergonomics are not always ideal. C23 somewhat improves the situation with <stdckdint.h>, a standardized version of GCC’s __builtin_add_overflow and friends. That has ergonomics issues too due to its verbosity, but at least it’s hard to screw up.
  - dllthomas 4y ago
    It can't (correctly) remove checks that would have prevented the undefined behavior, because that is changing the semantics of a program that (as written) does not trigger undefined behavior.
  - kaba0 4y ago
    That doesn’t sound like sound logic to me. The compiler assumes that at the point you have the arithmetics, they won’t overflow. This hinges upon all the former state of the program. If it can prove that the given integers won’t overflow (e.g. due to a previous, redundant check) then it can indeed remove a conditional, but the compiler can’t change the observable behavior of the program.
  - jefftk 4y ago
    The compiler is allowed to assume that if x and y are signed ints: if (x < 0 || y < 0) return -1; return x * y; Will not overflow. And if you try to check for overflow with: if (x < 0 || y < 0) return -1; if (x * y < 0) return -2; return x*y; Then yes, the compiler is within spec to remove your check because the only situation in which you could hit that check would be after signed integer overflow, which it is allowed to assume won't happen. One way to implement this check in GCC where the compiler will respect it would be: if (x < 0 || y < 0) return -1; int z; if (__builtin_smul_overflow(x, y, &z)) return -2; return z;

jenadine 4y ago

> dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens; Sure, but it didn't have to be like this. They could have said it is unspecified wihout allowing the compiler to assume it doesn't happen. Would C have been better if the spec was different?

wbl 4y ago
That's not what's happening. Because it's unspecified transforms that are safe in the absence of that behavior are safe to apply since they preserve the semantics. Even something like register allocation requires knowledge of what pointers point to.

marssaxman 4y ago

> But that's the thing: dereferencing an invalid pointer is undefined behaviour, which means the compiler is allowed to assume it never happens It would be more helpful for the compiler to assume that it does not know what will happen. This is how C actually worked for many years. > a C program executing undefined behaviour is _invalid C_. That was not formerly the case, and it is not always helpful to redefine C in this way. Sometimes you really are not trying to write portable code, and you really do want the behavior you know that the target machine will give you, even if the C spec doesn't require it.

account42 4y ago
That's OK, you can compile your program with -O0 if that's the behavior you want from your compiler.
- butlerm 4y ago
  There are many optimizations that a compiler can perform without relying on the optimization level to determine how to pervert your program that day. If different optimization levels produce different results that is bad thing, something to be avoided, not encouraged. If it is really necessary to generate random code when some anomalous situation is encountered, that should be a special option to enable dangerous non-deterministic if-you-made-a-mistake-we-will-delete-parts-of-your-program type behavior. I wouldn't consider that optimization though, more like disabling all your compiler's safety features.
  - gpderetta 4y ago
    Which optimizations?
    - butlerm 4y ago
      Loop unrolling for loops that have a static or range bounded number of iterations is a good example. Others include constant expression evaluation, dead code elimination, common subexpression elimination, and static function inlining.

somat 4y ago

Always missing in this argument is the logic of how to go from "undefined" to "can never happen". If the spec did not want it to happen they would have said "can not happen/illegal". but no, it is undefined by the spec. The spec knows that it can and will happen they just did not want to pin down the behavior of the compiler. So the compiler optimization team saying "we can assume this will never happen" is a blind almost maliciously complaint viewpoint.

aw1621107 4y ago
> Always missing in this argument is the logic of how to go from "undefined" to "can never happen". The reasoning is something like "if we assume UB doesn't happen, but it does happen, the resulting behavior is unpredictable. This is allowed by the standard, though, because UB allows for any behavior, including that produced by assuming UB doesn't happen." In other words, major implementations treat UB as preconditions. Violating those preconditions gets you Interesting Results (TM), but that's allowed by the standard because "unpredictable results " really means unpredictable results. For example, null pointer dereference is UB. If an implementation assumes null pointers can never be dereferenced, it can better optimize some code. If it turns out a null pointer is dereferenced, the argument is that whatever happens then is still permitted by the standard as the standard does not define any program semantics for programs containing UB.
uecker 4y ago
I agree that this does not follow from the wording in the standard and I am relatively sure that this was originally not implied. But this view point is repeated quite often nowadays. I think this is because prominent compiler developers promoted this point of view and used this for blaming the user ("Because you have UB in your program it is completely invalid. It is now ok that the compiler breaks it, and it is alone your fault".) The other response to your post is correct so, but this explanation would not allow UB to affect prior observable behavior.
gpderetta 4y ago
Ex falso quodlibet

mpweiher 4y ago

> is undefined behaviour, which means the compiler is allowed to assume it never happens No it's not. Or let me rephrase that. The standard says the following: Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). Which of these is "the compiler is allowed to assume it does not happen"?

User23 4y ago
> Which of these is "the compiler is allowed to assume it does not happen"? I agree with where you're coming from, but "ignoring the situation completely with unpredictable results" sounds like it pretty much fits the bill. Pretending something doesn't happen sounds a lot like ignoring it completely to me. What the standard obviously does exclude is nonsense like intentionally reformatting your drives.
- mpweiher 4y ago
  > Pretending something doesn't happen sounds a lot like ignoring it completely to me. Quite the opposite. Assuming it doesn't happen (not "pretending"), is very much not ignoring the situation, at least if you then act on that assumption that it does not happen. Ignoring it just lets it happen when it does, so if the program specifies an out-of-bounds access, the compiler generates code for an out-of-bounds access, ignoring the fact that it is an out of bounds access.
  - aw1621107 4y ago
    > Assuming it doesn't happen (not "pretending"), is very much not ignoring the situation, at least if you then act on that assumption that it does not happen. I'm not sure how assuming UB doesn't happen is distinguishable from a choosing to ignore the situation every time one comes up. You get the same result either way. For example, a compiler can assume that null pointers are never dereferenced, or every time a null pointer is/may be dereferenced it can just "ignore the situation" with the dereference. I'm not seeing a functional difference here. This is, of course, subject to the minor problem that "situation" is arguably underspecified. Compiler writers appear to interpret it as something akin to "code path" (so "ignoring the situation" means "ignoring code paths invoking UB"), while UB-goes-too-far proponents appear to interpret it more broadly, more like "the fact that UB can/will happen" (so "ignoring the situation" means "ignore the fact UB will/may happen"). > Ignoring it just lets it happen when it does, so if the program specifies an out-of-bounds access, the compiler generates code for an out-of-bounds access, ignoring the fact that it is an out of bounds access. Why wouldn't this fall under "behaving during translation or program execution in a documented manner characteristic of the environment" instead?

pygy_ 4y ago

> [whatever] is undefined behaviour, which means the compiler is allowed to assume it never happens; That interpretation is the root of the problem. Compilers authors use it to implement outright user hostile behavior in the name of elusive performance. Why would you spend resources looking for zero days when you can have a few LLVM contributors plant them in every program "as an optimization"?

Narishma 4y ago
So why not use a different compiler?
- pygy_ 4y ago
  Would you write it? You can use the same compiler with a language whose design committee isn't deliberately user hostile, like Rust (where UB-like behaviors in safe code are considered soundness bugs). https://runrust.miraheze.org/wiki/Undefined_Behavior https://runrust.miraheze.org/wiki/Undefined_Behavior
  - saagarjha 4y ago
    Rust makes guarantees for safe code that undefined behavior would violate. C(++) has no such mode and as such a comparison cannot be made.