> As stated there, I think there is room for a non-fwrapv approach that works just as well.
Maybe I am missing something, but I feel the compiler is doing exactly what you are suggesting in that subthread, and I replied in that vein. Perhaps I am misunderstanding what you are suggesting.
> I was probably unclear. I don't prefer unsigned to avoid the UB, but to get better generated assembly. While the unnecessary sign extension moves frequently come for free, occasionally they don't.
That seems backwards. Due to this "issue" the suggestion is that signed often generates better assembly, because the compiler can reason that overflow doesn't occur. With unsigned, the compiler must generate code that accommodates wraparound, since it is allowed to occur.
For example, I would have expected the example code in the link to compile to worse code if an unsigned loop counter is used, since wraparound is legal and so the compiler will have to zero-extend each value. As it turns out it doesn't, since of course the compiler can additionally prove that if wraparound occurs this particular loop won't even be entered (since the condition is `i < count`). This proof is quite fragile however, if the loop condition were different or if any other math occurs on index it would break as in [1].
So it is not the case that compiler exploitation of UB simply brings signed level with the performance of unsigned, but that it can make signed faster, and that's how it is usually presented/argued.
Now I'm certainly not saying it is always true that is signed is no slower than unsigned: signed has the advantage of no-wrapping semantics, while unsigned has the advantage of zero extension being free in some scenarios (on some platforms) since it can be rolled into some other operation which implicitly zero extends, which is what I think you were referring to [2]. Which one has an effect (or greater) effect depends on the code.
> I think the bugs are as likely to bite (or not bite) in either case, and need to be avoided by other means.
Definitely not. Code with unsigned is always just one subtraction away from a huge (nonsense) value. That issue is a real one that occurs all the time, and why even the C++ folks (who ISTM rarely admit errors) generally acknowledge that using unsigned for things like container sizes was an error (albeit an uncorrectable one).
Signed values on the other hand, only overflow if you actually have a gigantic value that exceeds 2^31 or 2^63 or whatever. In many cases such large values cannot occur due to range of possible values (i.e., the meaning of the values), and especially with 64-bit values are unlikely to occur outside of malice. Furthermore, unsigned values are still subject to this type of overflow - but at 2^32 (2^64) rather than 2^31 (2^63).
I agree though that you should still use other mechanisms to avoid overflow even with signed values, but there is benefit to defense in depth.
> I'm less sure of [compiler optimization having improved spectacularly]. You saw, for example, the current thread on RWT: https://www.realworldtech.com/forum/?threadid=182093&curpost... https://www.realworldtech.com/forum/?threadid=182093&curpost...? I think the situation is rather that recent compilers do a better job with naive code, but are more likely to pessimize consciously constructed code.
I'm not saying compilers are infallible. I am definitely in the camp that compilers have a long way to go. I believe optimization has improved dramatically in the last few decades. Perhaps "spectacularly" was too strong. This RWT example doesn't seem to contradict this: this is code that should have been well-optimized even 20 years ago. It is not in the set of things that needed to get better, so it should stay about the same. Due to some flaws in gcc, it actually has gotten worse. It is is large, complex code with lots of abstractions where optimization has gotten better.
You could try -O1 vs -O2 also for a kind of litmus test of the power of optimization. It's not an exact analogy, but -O1 code isn't that far off what older compilers could do, as many of the expensive optimizations enabled by -O2 are also the more recent ones (not coincidentally, since increasing hardware speed has made these palatable). On some code it makes no difference, and you'll even find cases where -O1 is faster. On lots of code -O2 is faster, and in some cases dramatically so (10x, etc).
In fact, if you run this example on x86, you'll probably find a big difference between -O1, -O2 (on clang) and -O3 (on gcc).
In any case, I don't think we even need to agree on whether optimization has improved, and by how much? It's not really central to the discussion.
We just need to agree that optimization is an important part of C/C++ compilers, valued by many users and a key criteria of evaluation between compilers. I think that is not controversial?
> I lack that belief, but I'm not sure if that makes me more or less cynical than you.
Keep in mind these discussions involve not just compiler writers but any interested parties, including people who strongly object to the current behavior. So even if don't believe the compiler writers can see a solution due to some type of institutional myopia, where are all the clever suggestions from outsiders? How come every time this comes up the most popular opinion is some solution-free observation about how insane this is rather than "this is obviously how it should be done"? I put it to you that it is because there is no great solution waiting in the wings, and the three most sane solutions (UB, wrapping, trapping) are already implemented and selectable by the user.
> I think the issue is that almost everyone involved in standards process is likely to share a set of core philosophical beliefs about programming that may be self-consistent but not cover the full range of possibility.
I think the standards committee is only tangentially involved here. The primary way the standards committee is involved is that they defined signed overflow as UB, but that was almost certainly because _different machines implemented signed arithmetic differently_, not out of some desire for the occasional no-overflow optimization.
Years later the compiler writers "abused" this to implement this type of optimization. So it is more about the compiler writers, IMO and whether their choice(s) are sane (see above).
The standards committee may become involved again, primary because they may remove the compiler's license to optimize this as UB by defining the behavior, and I hope to see that in my lifetime.
> It might not be the solution, but I think that what you are not seeing is that there are other interpretations of "implementation defined" besides "always wrap" or saturate.
Yes, I am not seeing them, but I am open to them! In fact I am practically begging for someone to explain them to me (in a patient and precise way, not going silent when it becomes clear that the nugget of the idea they had actually doesn't survive when fleshed out).
Also, to be precise, I do see other ideas, like:
- Trapping on overflow.
- Applying a heuristic to determine whether the user is actually anticipating overflow and allowing it to occur in that case, which would "fix" the overflow check case.
The first is already implemented and selectable, as far as I know, and the second seems likely to be worse than the other solutions, extremely controversial and very hard to implement.
> You (and I think the standards committees) seem to want all non-UB programs to be fully deterministic
On the part of the committee, I would say "definitely not", since otherwise they wouldn't have implementation defined behavior. From my part, I would say "Yes, that is obviously highly desirable!". I definitely want a deterministic, portable language. Anything else seems like a big mistake. Only C and C++ really deviate from this norm and mostly all in the name of performance. Weird behavior in the name of performance... somehow it seems like we are reversing our positions here?
One thing I'd like to see the standards committee do is have more implementation defined behavior, and feature-test macros to test it. The current implementation defined behavior is kind of useless to actually consume because you are supposed to hunt through the documentation for each compiler (if it even exists) and find what you want and hope it doesn't change in the future.
You could have fine-grained test macros, but it would even be great to have something like TWOS_COMPLEMENT_NOT_INSANE that means the platform has 8-bit bytes, two's complement signed and unsigned values, and everything behaves "as expected" in terms of conversions between them, etc.
Since pretty much every platform today has this, you could just have a:
#if !TWOS_COMPLEMENT_NOT_INSANE
#error "Sorry, this platform is to weird to support"
#endif
in your project and then basically write in a narrowed version of C/C++ which meets many more expectations of how things should work.
---
[1] https://godbolt.org/z/BLsXJ5 https://godbolt.org/z/BLsXJ5
[2] Other than this example, it is obvious that signed or unsigned can be slower than the other in cases where you are mixing and matching signedness within a loop in an inefficient way, especially when combined with mixing lengths (e.g., 32-bit vs 64-bit) as well.
> I think there's room for an approach that says "if you stray beyond these rules, the real processor will decide your fate" without the "electric fence of death" approach currently taken by UB. I might be wrong to call this "implementation defined", but I think that's where it best fits in.
How is this different from -fwrapv? The real processor implements the -fwrapv behavior, so it seems like the same thing.
If it's something different, how is it different than the current behavior of "compiler does whatever it wants and then the CPU executes the instructions"?
Maybe a specific example would help?
I think a key problem is that you are jumping ahead to "the real processor will decide your fate" - but that's not the interesting part. We all understand how real processors work (they wrap) - the whole question is what the language semantics of signed values are, because before you get to the processor, you have to compile the code. Even before that you have to write the code, and you want to know what happens when you do basic math on integers. If you want to invoke the specific processor behavior as part of the definition of how integers work, you also need to provide some kind of mapping from the source to machine operations (hint: I think -fwrapv behavior is the best or only way to do this practically).
> I don't want is for it to be a guaranteed no-op. Nor do I want it the compiler to eliminate code based on the assumption that the second call never happens. I want it to warn if it can (which can be a fatal warning if desired) but to always generate assembly that makes the potentially crashing call. My contract with the compiler is limited to determinism of the assembly it generates on my behalf, and does not include any guarantees or restrictions on the runtime behavior.
I see. What you basically want is a non-optimizing compiler, or a compiler that does only a limited amount of optimization following very strict rules (perhaps akin to something along the lines of how volatile is treated) about what can be optimized.
I think it is actually much more complicated than you would think (because the existing model of "as-if execution preserving required side-effects" is really very simple and clean).
More to the point, I don't think you'll ever get it [3] and you might not like what you get. 90% of optimization is eliminating code. The compiler eliminates function calls, assignments, math, allocations, jumps, branches, copies, constructors, and the list goes on forever. Elimination is the name of the game. Almost all of the time the result produced is indistinguishable from what you imagine the abstract machine doing, even though the combination of the compiler and real machine it is doing nothing like that at all.
These cases that get the press are notable for their exceptionality - at the intersection of a mismatch between what the standard says and how people imagine the language to work, and a similar mismatch between how people believe a compile (should?) work and how it does, and probably also a mismatch between the priorities of the compiler writers (performance on benchmarks) and users (correct code).
> My worry is that you (and others) don't consider this to be a standards compliant interpretation of "implementation defined", nor can it be "undefined behavior" if the compiler also reasons that UB can be assumed to never occur.
As above I'm not 100% clear on what you are suggesting, but I think it could be implementation defined as long as you can explain it (-fwrapv qualifies easily). After all, implementation defined has to be _defined_ in some way, right?
It could _definitely_ be UB, since UB can be whatever you want. A compiler could implement your suggestion right now and it would be within the rules. That just leaves us back where we are today though, right? Unknown or unexpected behavior in the presence of UB behavior.
I think there is a misconception that the "compiler assumes UB never occurs and removes all code that leads to UB". I think it's more accurate to say the compiler (in principle) just treats UB in the way that is most convenient (fast?) for the current context. In the specific case of a basic block that _always_ executes UB (e.g., references a pointer that is known to be not NULL), the most convenient/fast thing is probably to prune the code and paths that lead to this block - this type of thing happens all the time in non-buggy programs due to inlining, generic code, macros, auto-generated code, other optimization passes, etc.
In many other cases however, the most convenient thing to do is just to emit the instruction (or instruction sequence) that behaves in the required way for non-UB code, and behaves "as it will" on the hardware for UB code, like all the stuff around shifts, division by zero and even various builtins and intrinsics. The compiler doesn't go out of its way to remove the UB code, it just does what is fastest.
Nobody complains about the latter behavior though, but of course it leads to bugs all the same - but people blame the programmer, not the compiler in this case, because the result is more "obvious" (I think).
---
[3] The niche for a compiler that executes what you write more or less verbatim is just too small (beyond what you already get with -Og or -O0 or whatever): people write in C/C++ for speed, if you wanted something slower you have many better, safer options available.