8 ms·

The way the compilers abuse undefined behavior to "optimize" away sanity checks we explicitly put in is just downright insane to me

tlb 2y ago
Clang has a mode (UBsan) that flags and optionally aborts if you execute UB. It runs several times slower because it has to check lots of things, but it might make sense for some programs.
- AlotOfReading 2y ago
  UBSan is not exhaustive. There are large and important cases of UB it cannot detect. There are also cases where the compiler simply won't insert the proper checks even for detectable issues. None of the sanitizers are broadly intended for production use either, though UBSan offers a specific mode you can select (fsanitize-minimal-runtime) that's appropriate for some production uses.
- lidavidm 2y ago
  GCC can also use UBSan (and ASan) [1], and furthermore, it shouldn't be "several times slower" (are you thinking of Valgrind?). Clang itself describes the checks as "small runtime cost" [2] and there's a minimal variant as well. [1]: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.h... [2]: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
kccqzy 2y ago
It's more insane to me that the people who are putting in these "sanity checks" don't know enough about the language to find out that their "sanity checks" are both wrong and useless in the first place.
- tlb 2y ago
  Many sanity checks are preserved with -O0, so they're still useful in debugging. And they serve as documentation of expected conditions. I often put in checks that I expect to be statically eliminated to remind myself what is supposed to be true.
- shadowgovt 2y ago
  The working draft standard for C++ (2022-12-18, document N4928) is 2,110 pages long at a reasonable-sized font. I'd actually go the other way: I think most practicing programmers have not read the entire standard specifying most languages they use day-to-day and have no real idea what the abstraction-break looks like that turns their code into a format consumable by the next layer down. How many Java programmers do we assume know anything about the bytecode of the JVM, for example?
  - pjmlp 2y ago
    Which includes the whole standard library. The corresponding Java, C#, Python standards, equally with the standard library, are even bigger. To come back to the point, many don't know how deceptively complex Python happens to be, even though on the surface looks like a BASIC replacement.
    - int_19h 2y ago
      To be fair, even if you look at just the language alone, C++ is still extremely large. But the real problem is when you combine this sheer size with the amount of subtle footguns. If we compare to equivalent Python code, for example, the behavior is simple and straightforward: division by zero causes a runtime error that is reported in a well-defined manner. Similarly with other arithmetic issues in C++ that can trigger UB - e.g. integer overflow is just not a thing in Python (short of OOM). The detailed rules may well be complicated, but it doesn't matter as much when the behavior is intuitive and conforms to common sense expectations.
      - kccqzy 2y ago
GrantMoyer 2y ago
Undefined behavior isn't a threat; it's a promise. That came out wrong. What I mean is that it's not a threat from the standard that its buddy the compiler will ruin your computer, your day, and your life. Instead, it's a promise from the programmer to the compiler that the programmer won't perform some operations. Since the programmer promised not to do some things, the compiler assumes those things aren't done while reasoning about the code, but if it turns out one of those things was done, the compiler's whole chain of reasoning is potentially invalid. So the compiler's options are to trust the programmer, and make the assumptions, or to instead treat the programmer as a dirty liar, and not make any of the assumptions while expending much effort considering all the subtle ways the programmer could try to trick it. Actually, there's a third option, trust but verify, where the compiler trusts the programmer to make a best effort, but verifies the assumptions when convenient. Unfortunately, it's not often convenient to verify in C or C++, but broadly compilers are getting better at it over time.
- pif 2y ago
  Very well put! I'll just add that the third option is usually not considered viable because the main point of using C/C++ is for performance, thus people writing code want their compiler to optimize it as much as possible.
  - int_19h 2y ago
    In practice, the people pushing the most for UB being a "license to kill" tend to be compiler implementors though, not their users.
    - GrantMoyer 2y ago
      I don't think compilers consider undefined behavior a license to kill; I think the problem is genuinely hard. The priciple of explosion[1] tells us that any wrong assumption, even the most innocuous looking, can lead to an unboundedly wrong conclusion. The more complex your graph of logical deductions, the worse the problem gets (and good optimizations require a lot of complex logic). It may be possible to limit the bad effects of faulty assumptions by being very careful about how deductions propogate from unreliable assumptions. However it's certainly not easy, especially without throwing out all optimizations that interact with a potentially undefined operation. It is possible to design a languge such that a compiler doesn't need to make assumptions to perform optimizations, if programs are required to give enough information to compilers that they can statically prove properties they need to perform optimizations. But C and C++ are not designed that way, and it can't easily be retrofitted on top of them. [1]: https://en.wikipedia.org/wiki/Principle_of_explosion https://en.wikipedia.org/wiki/Principle_of_explosion
pjc50 2y ago
I'm coming round to the idea that entirely eliminating lines of code in the optimizer is something that should be a warning instead, to be fed back to the user. (NB entirely. That is, it's reasonable to coalesce arithmetic expressions together or inline functions such that a single operation in the output has subsumed multiple lines of code. However, if for a source S you can remove one or more entire lines of code to produce source S' with the same output, then something is wrong with the intent of the program or the compiler has misinterpreted it)
- UncleMeat 2y ago
  This would produce at least one warning for almost every nontrivial function in your program, even with your concession towards simplifying arithmetic expressions.
  - ben_w 2y ago
    Can you give an example? This is surprising, and I can't imagine such… but then, I never wrote a compiler or needed to examine the assembly from one.
    - UncleMeat 2y ago
      Copy elision and dead store elimination are extremely common examples. Redundant bounds check elimination on containers is another. I'd consider hoisting outside of a loop also to be eliminating code. I suppose some of this depends on what you mean by "eliminating a line of code", as an optimizing compiler pretty quickly transforms the program into a state where any kind of 1:1 mapping between lines of code and instructions is impossible.
      - ben_w 2y ago
        I suspect I have misunderstood the point you were making in the previous comment, are not the first two already things one can get warnings for, and the third a useful thing to know you're doing redundantly? I don't think hoisting would be covered by this suggestion, pjc50 suggested deletion rather than moving. I read the proposal very literally, which also means I'm unclear what options you have in your final paragraph: Compile function as is; For each line, make a copy with that line missing and recompile; when this identical to the original, flag that line as "warning: optimised out of existence"
UncleMeat 2y ago
It cannot optimize away correct sanity checks. The sanity checks that it removes are ones that attempt to detect UB after it has occurred. If you instead make sure that checking for UB is checking before the dangerous operation then it is a correct program and the compiler won't surprise you.
- lelanthran 2y ago
  > The sanity checks that it removes are ones that attempt to detect UB after it has occurred. The point is that there is still value in that - your running program can at least say "oops" after the fact. You'll at least get errors when running your tests! Removing the `printf ("oops")` altogether under the guise of "well, something 3 lines above was UB, so we can delete the rest of the program" is what users are complaining about. The way the compilers go about it now, is that they don't even let you fail tests because you made an oopsie a few lines ago and now everything after that is considered safe for deletion. Look at this example: char *myvar = null; ... myvar[0] = 'A'; ... if (myvar == null) { printf ("oops\n"); } I literally want every single one of those lines emitted in the final output. 1. I don't want the 'myvar[0]' assignment removed because the compiler determined that mvar is NULL. If it's there, at least I'll get a crash before the 'oops'. If the crash doesn't come, at least I have the 'oops' to warn in testing. 2. I don't want the `myvar == null` check removed. If the crash in the assignment didn't happen, I **want** the message 'oops' printed. The compiler is clearly going against the intention of the programmer if it fails to emit code for any of those lines, because it turns a failing test into a passing one.
  - UncleMeat 2y ago
    The challenge is that a compiler cannot really provide consistent "after UB occurs" behavior that you can rely on for these sorts of post-facto checks without pessimizing behavior in pretty bad ways in a lot of situations. If you want to be able to observe values through invalid pointers then the compiler can't do things like store locals in registers since maybe you care about checking a value through one of these pointers. Huge mess! Like, whose to say that the line "myvar[0] = 'A';" doesn't write to the location in memory where the constant string "oops\n" is stored? How will the compiler ensure that your program does what you expect after a bogus write somewhere in memory?
pif 2y ago
If the compiler threw it away, then the check was ill formed or just arrived too late.
quelsolaar 2y ago
Id recomend watching this video on how UB works in C. https://www.youtube.com/watch?v=w3_e9vZj7D8 https://www.youtube.com/watch?v=w3_e9vZj7D8 Compilers generaly do not abuse UB (outside of compiler bugs), its just that UB is a very missunderstood subject.
- gavinhoward 2y ago
  Not GP. I watched the video and didn't learn anything new. I know UB pretty well. > Compilers generaly do not abuse UB It's disheartening to see this take from the guy who said that one word broke C. It is also false, in my experience because some people, including compiler developers, think that compiler freedom is the definition of UB, not a side effect of that one-word change: https://gavinhoward.com/2023/08/the-scourge-of-00ub/ https://gavinhoward.com/2023/08/the-scourge-of-00ub/ So your claim in the video that such things are scary to compiler developers is sad. They have broken existing code; they are not afraid to. I used to be somewhat confident (https://gavinhoward.com/2024/05/a-grateful-open-letter-to-jeanheyd-meneide/ https://gavinhoward.com/2024/05/a-grateful-open-letter-to-je...) in the direction of C with you and JeanHeyd Meneide on WG14. But if you sincerely believe what you just said, then it seems you have changed your mind since you wrote that one word changed C. So I am not as confident anymore.
  - quelsolaar 2y ago
    My "one word broke C" article is wrong in all kinds of ways. I understand C way better than i used to. The things that need fixing in C are not UB, its very complex memory model stuff. I have no interest in changing any fundamentals or adding new features.
    - gavinhoward 2y ago
      It is sad to see your transformation. 00UB does need fixing, and C needs less UB. If you don't think so anymore, then C will become untenable for me to use. > I understand C way better than i used to. Maybe, but I did not see it in that video. What I do know is that you do not know much about the humans using C or the environments where C is used. C is a tool for humans, not machines. Prioritizing machines over humans will make C worse over time, and implementors will use that to further excuse their behavior.

tlb 2y ago

Clang has a mode (UBsan) that flags and optionally aborts if you execute UB. It runs several times slower because it has to check lots of things, but it might make sense for some programs.

AlotOfReading 2y ago
UBSan is not exhaustive. There are large and important cases of UB it cannot detect. There are also cases where the compiler simply won't insert the proper checks even for detectable issues. None of the sanitizers are broadly intended for production use either, though UBSan offers a specific mode you can select (fsanitize-minimal-runtime) that's appropriate for some production uses.
lidavidm 2y ago
GCC can also use UBSan (and ASan) [1], and furthermore, it shouldn't be "several times slower" (are you thinking of Valgrind?). Clang itself describes the checks as "small runtime cost" [2] and there's a minimal variant as well. [1]: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.h... [2]: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

kccqzy 2y ago

It's more insane to me that the people who are putting in these "sanity checks" don't know enough about the language to find out that their "sanity checks" are both wrong and useless in the first place.

tlb 2y ago
Many sanity checks are preserved with -O0, so they're still useful in debugging. And they serve as documentation of expected conditions. I often put in checks that I expect to be statically eliminated to remind myself what is supposed to be true.
shadowgovt 2y ago
The working draft standard for C++ (2022-12-18, document N4928) is 2,110 pages long at a reasonable-sized font. I'd actually go the other way: I think most practicing programmers have not read the entire standard specifying most languages they use day-to-day and have no real idea what the abstraction-break looks like that turns their code into a format consumable by the next layer down. How many Java programmers do we assume know anything about the bytecode of the JVM, for example?
- pjmlp 2y ago
  Which includes the whole standard library. The corresponding Java, C#, Python standards, equally with the standard library, are even bigger. To come back to the point, many don't know how deceptively complex Python happens to be, even though on the surface looks like a BASIC replacement.
  - int_19h 2y ago
    To be fair, even if you look at just the language alone, C++ is still extremely large. But the real problem is when you combine this sheer size with the amount of subtle footguns. If we compare to equivalent Python code, for example, the behavior is simple and straightforward: division by zero causes a runtime error that is reported in a well-defined manner. Similarly with other arithmetic issues in C++ that can trigger UB - e.g. integer overflow is just not a thing in Python (short of OOM). The detailed rules may well be complicated, but it doesn't matter as much when the behavior is intuitive and conforms to common sense expectations.
    - kccqzy 2y ago

GrantMoyer 2y ago

Undefined behavior isn't a threat; it's a promise. That came out wrong. What I mean is that it's not a threat from the standard that its buddy the compiler will ruin your computer, your day, and your life. Instead, it's a promise from the programmer to the compiler that the programmer won't perform some operations. Since the programmer promised not to do some things, the compiler assumes those things aren't done while reasoning about the code, but if it turns out one of those things was done, the compiler's whole chain of reasoning is potentially invalid. So the compiler's options are to trust the programmer, and make the assumptions, or to instead treat the programmer as a dirty liar, and not make any of the assumptions while expending much effort considering all the subtle ways the programmer could try to trick it. Actually, there's a third option, trust but verify, where the compiler trusts the programmer to make a best effort, but verifies the assumptions when convenient. Unfortunately, it's not often convenient to verify in C or C++, but broadly compilers are getting better at it over time.

pif 2y ago
Very well put! I'll just add that the third option is usually not considered viable because the main point of using C/C++ is for performance, thus people writing code want their compiler to optimize it as much as possible.
- int_19h 2y ago
  In practice, the people pushing the most for UB being a "license to kill" tend to be compiler implementors though, not their users.
  - GrantMoyer 2y ago
    I don't think compilers consider undefined behavior a license to kill; I think the problem is genuinely hard. The priciple of explosion[1] tells us that any wrong assumption, even the most innocuous looking, can lead to an unboundedly wrong conclusion. The more complex your graph of logical deductions, the worse the problem gets (and good optimizations require a lot of complex logic). It may be possible to limit the bad effects of faulty assumptions by being very careful about how deductions propogate from unreliable assumptions. However it's certainly not easy, especially without throwing out all optimizations that interact with a potentially undefined operation. It is possible to design a languge such that a compiler doesn't need to make assumptions to perform optimizations, if programs are required to give enough information to compilers that they can statically prove properties they need to perform optimizations. But C and C++ are not designed that way, and it can't easily be retrofitted on top of them. [1]: https://en.wikipedia.org/wiki/Principle_of_explosion https://en.wikipedia.org/wiki/Principle_of_explosion

pjc50 2y ago

I'm coming round to the idea that entirely eliminating lines of code in the optimizer is something that should be a warning instead, to be fed back to the user. (NB entirely. That is, it's reasonable to coalesce arithmetic expressions together or inline functions such that a single operation in the output has subsumed multiple lines of code. However, if for a source S you can remove one or more entire lines of code to produce source S' with the same output, then something is wrong with the intent of the program or the compiler has misinterpreted it)

UncleMeat 2y ago
This would produce at least one warning for almost every nontrivial function in your program, even with your concession towards simplifying arithmetic expressions.
- ben_w 2y ago
  Can you give an example? This is surprising, and I can't imagine such… but then, I never wrote a compiler or needed to examine the assembly from one.
  - UncleMeat 2y ago
    Copy elision and dead store elimination are extremely common examples. Redundant bounds check elimination on containers is another. I'd consider hoisting outside of a loop also to be eliminating code. I suppose some of this depends on what you mean by "eliminating a line of code", as an optimizing compiler pretty quickly transforms the program into a state where any kind of 1:1 mapping between lines of code and instructions is impossible.
    - ben_w 2y ago
      I suspect I have misunderstood the point you were making in the previous comment, are not the first two already things one can get warnings for, and the third a useful thing to know you're doing redundantly? I don't think hoisting would be covered by this suggestion, pjc50 suggested deletion rather than moving. I read the proposal very literally, which also means I'm unclear what options you have in your final paragraph: Compile function as is; For each line, make a copy with that line missing and recompile; when this identical to the original, flag that line as "warning: optimised out of existence"

UncleMeat 2y ago

It cannot optimize away correct sanity checks. The sanity checks that it removes are ones that attempt to detect UB after it has occurred. If you instead make sure that checking for UB is checking before the dangerous operation then it is a correct program and the compiler won't surprise you.

lelanthran 2y ago
> The sanity checks that it removes are ones that attempt to detect UB after it has occurred. The point is that there is still value in that - your running program can at least say "oops" after the fact. You'll at least get errors when running your tests! Removing the `printf ("oops")` altogether under the guise of "well, something 3 lines above was UB, so we can delete the rest of the program" is what users are complaining about. The way the compilers go about it now, is that they don't even let you fail tests because you made an oopsie a few lines ago and now everything after that is considered safe for deletion. Look at this example: char *myvar = null; ... myvar[0] = 'A'; ... if (myvar == null) { printf ("oops\n"); } I literally want every single one of those lines emitted in the final output. 1. I don't want the 'myvar[0]' assignment removed because the compiler determined that mvar is NULL. If it's there, at least I'll get a crash before the 'oops'. If the crash doesn't come, at least I have the 'oops' to warn in testing. 2. I don't want the `myvar == null` check removed. If the crash in the assignment didn't happen, I **want** the message 'oops' printed. The compiler is clearly going against the intention of the programmer if it fails to emit code for any of those lines, because it turns a failing test into a passing one.
- UncleMeat 2y ago
  The challenge is that a compiler cannot really provide consistent "after UB occurs" behavior that you can rely on for these sorts of post-facto checks without pessimizing behavior in pretty bad ways in a lot of situations. If you want to be able to observe values through invalid pointers then the compiler can't do things like store locals in registers since maybe you care about checking a value through one of these pointers. Huge mess! Like, whose to say that the line "myvar[0] = 'A';" doesn't write to the location in memory where the constant string "oops\n" is stored? How will the compiler ensure that your program does what you expect after a bogus write somewhere in memory?

pif 2y ago

If the compiler threw it away, then the check was ill formed or just arrived too late.

quelsolaar 2y ago

Id recomend watching this video on how UB works in C. https://www.youtube.com/watch?v=w3_e9vZj7D8 https://www.youtube.com/watch?v=w3_e9vZj7D8 Compilers generaly do not abuse UB (outside of compiler bugs), its just that UB is a very missunderstood subject.

gavinhoward 2y ago
Not GP. I watched the video and didn't learn anything new. I know UB pretty well. > Compilers generaly do not abuse UB It's disheartening to see this take from the guy who said that one word broke C. It is also false, in my experience because some people, including compiler developers, think that compiler freedom is the definition of UB, not a side effect of that one-word change: https://gavinhoward.com/2023/08/the-scourge-of-00ub/ https://gavinhoward.com/2023/08/the-scourge-of-00ub/ So your claim in the video that such things are scary to compiler developers is sad. They have broken existing code; they are not afraid to. I used to be somewhat confident (https://gavinhoward.com/2024/05/a-grateful-open-letter-to-jeanheyd-meneide/ https://gavinhoward.com/2024/05/a-grateful-open-letter-to-je...) in the direction of C with you and JeanHeyd Meneide on WG14. But if you sincerely believe what you just said, then it seems you have changed your mind since you wrote that one word changed C. So I am not as confident anymore.
- quelsolaar 2y ago
  My "one word broke C" article is wrong in all kinds of ways. I understand C way better than i used to. The things that need fixing in C are not UB, its very complex memory model stuff. I have no interest in changing any fundamentals or adding new features.
  - gavinhoward 2y ago
    It is sad to see your transformation. 00UB does need fixing, and C needs less UB. If you don't think so anymore, then C will become untenable for me to use. > I understand C way better than i used to. Maybe, but I did not see it in that video. What I do know is that you do not know much about the humans using C or the environments where C is used. C is a tool for humans, not machines. Prioritizing machines over humans will make C worse over time, and implementors will use that to further excuse their behavior.