4 ms·

with all due respect, my friends who work on Clang and GCC are better programmers than 99% of commenters on this site I presume that both you and the majority

by nkurz 8y ago

with all due respect, my friends who work on Clang and GCC are better programmers than 99% of commenters on this site I presume that both you and the majority of the people who work on compilers are great programmers. Which makes difficulty in communication even stranger. Summarizing the anti-UB side, we really don't like it when the compiler drops explicit error checking code on the basis that error check would only be used if undefined behavior occurred. That is, I'd like a naive check for overflow to either emit an error message, or to compile as written with the behavior dependent on the underlying architecture. Even register allocation depends on exploiting undefined behavior—who's to say that some wild pointer you created didn't accidentally point to some local variable? The confusing part is that you (and many other smart people) seem to feel that "don't drop explicit error checks" is completely equivalent to "don't store local variables in registers". To me (to us) these are completely independent, with the dropped errors checking being a "real problem" and the possibly clobbered variables being a "not a problem", and in fact, everything working the way it should. Could you explain more why you feel these two are equivalent? My guess is that the difference is philosophical, between those who feel that the source code is a recipe, versus those who feel the source code is a mathematical proof. In my view, the goal of the compiler isn't to create a perfect program suitable for an perfect abstract machine, rather it's to translate the intent of the programmer into assembly language suitable for a particular deterministic processor. I have no problem with a compiler that produces code that crashes on malformed input (division by zero, reading a NULL pointer) but I want it to leave the actual crash up to the processor. With overflow as with the case of clobbered aliased variables, I'm fine if the compiler generates code that only works correctly if the runtime error does not occur, but I'm not OK with reasoning non-locally that the error checks can be omitted because the error is logically impossible. Do you really not distinguish these two cases?

BeeOnRope 8y ago
Dropping "explicit error checks" is no doubt bad from the user's point of view, but the primary difficulty is that to the compiler, working at the abstract level of multi-pass optimization of the IR, the good and bad cases all look the same. Pruning paths that lead to UB leads to some significant actual real-world optimizations (and these extend beyond these various signed-wrapping examples), and people want those whether they know it or not. Unfortunately, to the compilers, your overflow check just looks like another one of these cases. That is, it is very hard for the compiler to guess the programmer's intent, and say "oh, this is actually a check which is trying to avoid overflow, and despite that it uses UB, we should compile this part with wrapping semantics instead", and yet apply the optimization in all the other places you would want it. You could probably come up with some type of pattern recognition, or some heuristics like "if the optimization makes some of the original source code dead unconditionally, don't make it", but I think this is both very difficult and in many cases worse than the current disease. I think you can make a strong argument that the optimizations enabled by some types of UB exploitation are not worth the bugs and vulnerabilities they cause, and so the UB should be removed entirely, making it defined or implementation defined. Trying to have your cake (yes, it should still make these optimizations x and y) but eating it too (oh, but case z is different, see?) is a tougher proposition.
- nkurz 8y ago
  Dropping "explicit error checks" is no doubt bad from the user's point of view Only from the user's point of view? No, I'd say it's bad in some absolute sense as well. Can we at least agree that it would be better for the compiler to do what the programmer clearly intended? Yes, given current limitations, it can be difficult to see how a compiler has enough information to always recognize this intent. But I hope we at least both agree that if all else doing equal, doing what the programmer meant is the preferred choice? As a parallel, I think the approach of having -Wall flag "if (a = b)" as a likely error but be overrideable with "if ((a = b))" is a great solution. Maybe this one can be solved with an extra block, or with a better #pragma, or just really good heuristics. But the current approach of silent deletion of explicit code for undefined behavior is a terrible pattern!
  - BeeOnRope 8y ago
    > Only from the user's point of view? No, I'd say it's bad in some absolute sense as well. I didn't say _only_, but I'm not sure about "bad" an an "absolute" sense. I think it would take a lot of defining the exact question to debate that. For me if it is a bad outcome from the user's point of view, it is already worth discussing. > Can we at least agree that it would be better for the compiler to do what the programmer clearly intended? I guess? It would be pretty neat to work in a language that did what I intended rather than what I wrote, but it will quickly become unsustainable when two people write the same code but intended different things. That problem doesn't apply exactly here since the code is just UB, so the intent doesn't exactly conflict with the actual semantics since there are no semantics. However, the compiler writers have decided to chose the behavior which produces the fastest code. > Maybe this one can be solved with an extra block, or with a better #pragma, or just really good heuristics. IMO, it is largely already solved. If you want signed overflow to behave as, well, signed overflow, use -fwrapv. You pay the small price in reduced optimization opportunities, but you get well-defined behavior on wrapping always (note: wrapping is still almost always a bug!). I think you can make a good argument that this flag should be the default behavior. Certainly, if you were designing a new language you might want to make it the default and maybe C and/or C++ will get there some day. > But the current approach of silent deletion of explicit code for undefined behavior is a terrible pattern! This sentence embodies my main complaint with the endless discussions on this topic. There is no optimization that just removes signed overflows checks, inserted there by a malicious compiler writer. There is a big family of interlocking optimizations, which as a whole are responsible for some spectacular gains in compiled code efficiency. For example there is dead code elimination, which is really important. There is treating code that does something impossible or UB as dead, which is also important (it extends way beyond this signedness stuff). All this conspires to eliminate the _explicit_ overflow check which is the poster-child for compiler misbehavior - but the same optimizations do all sorts of other stuff that you wouldn't have a problem with. Everyone points there finger at this case that looks bad and says "obviously don't do that", but very few explain how. Deep within the guts of the optimizer, you just have IR which probably looks nothing like the code you wrote, there is no distinction between explicit and other code, and no "annotation of intent" to tell anyone what the programmer _really meant_. You have most or all of the major compilers implementing this optimization, yet none of them have apparently solved this simple case of not messing up what the programmer wrote? Probably because it is not at all simple. The solution I'm aware of is to just always treat overflow as defined. I.e., the -fwrapv solution. I'd be fine if compiler makes made it the default - but I guess the competitive pressures mean that compilers generally leave all legal optimizations on at the highest levels, to not be slower than the other guy. I'm OK with that also - at least it's a consistent rule. The compiler-writers position is if you write broken code that invokes UB, you don't get the complain. I think that's often a reasonable approach. In other languages it's just the obvious default: it's just a bit annoying in C/C++ because there are so many things that people think should work, and did work, but are actually UB.

by nkurz 8y ago

BeeOnRope 8y ago

Dropping "explicit error checks" is no doubt bad from the user's point of view, but the primary difficulty is that to the compiler, working at the abstract level of multi-pass optimization of the IR, the good and bad cases all look the same. Pruning paths that lead to UB leads to some significant actual real-world optimizations (and these extend beyond these various signed-wrapping examples), and people want those whether they know it or not. Unfortunately, to the compilers, your overflow check just looks like another one of these cases. That is, it is very hard for the compiler to guess the programmer's intent, and say "oh, this is actually a check which is trying to avoid overflow, and despite that it uses UB, we should compile this part with wrapping semantics instead", and yet apply the optimization in all the other places you would want it. You could probably come up with some type of pattern recognition, or some heuristics like "if the optimization makes some of the original source code dead unconditionally, don't make it", but I think this is both very difficult and in many cases worse than the current disease. I think you can make a strong argument that the optimizations enabled by some types of UB exploitation are not worth the bugs and vulnerabilities they cause, and so the UB should be removed entirely, making it defined or implementation defined. Trying to have your cake (yes, it should still make these optimizations x and y) but eating it too (oh, but case z is different, see?) is a tougher proposition.

nkurz 8y ago
Dropping "explicit error checks" is no doubt bad from the user's point of view Only from the user's point of view? No, I'd say it's bad in some absolute sense as well. Can we at least agree that it would be better for the compiler to do what the programmer clearly intended? Yes, given current limitations, it can be difficult to see how a compiler has enough information to always recognize this intent. But I hope we at least both agree that if all else doing equal, doing what the programmer meant is the preferred choice? As a parallel, I think the approach of having -Wall flag "if (a = b)" as a likely error but be overrideable with "if ((a = b))" is a great solution. Maybe this one can be solved with an extra block, or with a better #pragma, or just really good heuristics. But the current approach of silent deletion of explicit code for undefined behavior is a terrible pattern!
- BeeOnRope 8y ago
  > Only from the user's point of view? No, I'd say it's bad in some absolute sense as well. I didn't say _only_, but I'm not sure about "bad" an an "absolute" sense. I think it would take a lot of defining the exact question to debate that. For me if it is a bad outcome from the user's point of view, it is already worth discussing. > Can we at least agree that it would be better for the compiler to do what the programmer clearly intended? I guess? It would be pretty neat to work in a language that did what I intended rather than what I wrote, but it will quickly become unsustainable when two people write the same code but intended different things. That problem doesn't apply exactly here since the code is just UB, so the intent doesn't exactly conflict with the actual semantics since there are no semantics. However, the compiler writers have decided to chose the behavior which produces the fastest code. > Maybe this one can be solved with an extra block, or with a better #pragma, or just really good heuristics. IMO, it is largely already solved. If you want signed overflow to behave as, well, signed overflow, use -fwrapv. You pay the small price in reduced optimization opportunities, but you get well-defined behavior on wrapping always (note: wrapping is still almost always a bug!). I think you can make a good argument that this flag should be the default behavior. Certainly, if you were designing a new language you might want to make it the default and maybe C and/or C++ will get there some day. > But the current approach of silent deletion of explicit code for undefined behavior is a terrible pattern! This sentence embodies my main complaint with the endless discussions on this topic. There is no optimization that just removes signed overflows checks, inserted there by a malicious compiler writer. There is a big family of interlocking optimizations, which as a whole are responsible for some spectacular gains in compiled code efficiency. For example there is dead code elimination, which is really important. There is treating code that does something impossible or UB as dead, which is also important (it extends way beyond this signedness stuff). All this conspires to eliminate the _explicit_ overflow check which is the poster-child for compiler misbehavior - but the same optimizations do all sorts of other stuff that you wouldn't have a problem with. Everyone points there finger at this case that looks bad and says "obviously don't do that", but very few explain how. Deep within the guts of the optimizer, you just have IR which probably looks nothing like the code you wrote, there is no distinction between explicit and other code, and no "annotation of intent" to tell anyone what the programmer _really meant_. You have most or all of the major compilers implementing this optimization, yet none of them have apparently solved this simple case of not messing up what the programmer wrote? Probably because it is not at all simple. The solution I'm aware of is to just always treat overflow as defined. I.e., the -fwrapv solution. I'd be fine if compiler makes made it the default - but I guess the competitive pressures mean that compilers generally leave all legal optimizations on at the highest levels, to not be slower than the other guy. I'm OK with that also - at least it's a consistent rule. The compiler-writers position is if you write broken code that invokes UB, you don't get the complain. I think that's often a reasonable approach. In other languages it's just the obvious default: it's just a bit annoying in C/C++ because there are so many things that people think should work, and did work, but are actually UB.