13 ms·

Undefined Behavior Is Really Undefined

94 points by erwan 8y ago

saagarjha 8y ago
There’s really no need to pass -O9 to GCC. Anything over -O3 should become -O3 anyways.
- jibal 8y ago
  It doesn't hurt.
  - jnurmine 8y ago
    I wonder if it does anything useful at all, since it -O9 is not defined, unless the compiler is some kind of special version.
    - yjftsjthsd-h 8y ago
      IIRC, the code is actually `if O>2` or so; 3 and up are literally identical. EDIT: Per https://wiki.gentoo.org/wiki/GCC_optimization#What_about_-O_levels_higher_than_3.3F https://wiki.gentoo.org/wiki/GCC_optimization#What_about_-O_... it's `enabled = (level >= 3);`
  - johnisgood 8y ago
    It puts the article in a different light. Makes you wonder what else he is incompetent or negligent about. In this sense it does hurt: it hurts him, it makes him less credible.
    - baby 8y ago
      you realize who the author of the article is do you?
      - johnisgood 8y ago
        What are you trying to say? That because he is an authority figure to you and perhaps to other people, it is somehow OK to be ignorant about certain things? Last time I did something similar it was due to sheer ignorance and placebo. It might be a bad mistake, or a terrible habit, but it is what it is. It is silly regardless of who it comes from. :)
        baby 8y ago
        He is competent, and he wrote a blogpost to share something he knows on the web. It's a bit goofy to start claiming things about a character you don't know from a free publication.
        johnisgood 8y ago
        Let me repeat: it is silly regardless of who it comes from. Everything else is irrelevant. You appear to be biased to the point of completely ignoring everything I have said. I get it, you like him. It does not change anything. I merely suggested the possibility that he might be negligent or wrong about other relevant things. Do you think that this is impossible?
        jibal 8y ago
        The author is competent and not at all silly. And the author explicitly stated what -O9 does and why he used it. > You appear to be biased to the point of completely ignoring everything I have said. This is in violation of you usage agreement with ycombinator. > I merely suggested the possibility that he might be negligent or wrong about other relevant things. No, that's not true. > Do you think that this is impossible? Strawman. I see that you created your account just a few days ago ... you might want to be careful with it. In any case, I won't be engaging with you further, and I doubt that I'm the only one.
        johnisgood 8y ago
        > This is in violation of you usage agreement with ycombinator. I do not see how it would be the case. > No, that's not true. Care to elaborate as to why you think that it is impossible for him to be negligent or wrong? > I see that you created your account just a few days ago What does it have to do with anything? > you might want to be careful with it Are you threatening me? > I won't be engaging with you Okay. > I doubt that I'm the only one Perhaps.
- pornin 8y ago
  In older times (around the GCC 2.7 series, I think), "-O9" was documented to be the "future-proof" setting that selects the maximum optimization level, whatever it is. In practice, GCC never defined anything beyond -O3, but I still use -O9 when I want to make GCC do its best (or worst). Call it force of habit from a lengthening experience. (With Clang I use -O3 because if I write -O9 it screams at me.) (Note that -O3 is usually not that good an idea: aggressive loop unrolling can sometimes decrease performance because of cache issues, for instance. To optimize code properly, you have to make benchmarks and adjust both the code and compiler flags accordingly, in a well-thought feedback cycle. The use of aggressive optimizations in the blog post is to make the examples trip UB more clearly.)
kibwen 8y ago
Upvoted for the signed integer overflow example. I'll admit that I actually don't know the most idiomatic, bulletproof way of testing for signed overflow in C; if you google "how to test signed integer overflow in C", the very first result is essentially equivalent to the buggy example in the blog post ( https://www.geeksforgeeks.org/check-for-integer-overflow/ https://www.geeksforgeeks.org/check-for-integer-overflow/ ), and I'm not keen to repeat the legendary case of signed overflow within the PHP interpreter: https://web.archive.org/web/20120412194929/http://use.perl.org/use.perl.org/_Aristotle/journal/33448.html https://web.archive.org/web/20120412194929/http://use.perl.o...
- 0xcde4c3db 8y ago
  I'm not sure there is a reliable, generic approach for detecting overflow in conformant C; you probably have to come up with the appropriate proof for the specific operation that you're about to do. OpenBSD does it for multiplication in reallocarray() [1], which was introduced because it was easy to code an integer overflow when doing something like malloc(sizeof(member) * nmembers). [1] https://github.com/openbsd/src/blob/master/lib/libc/stdlib/reallocarray.c https://github.com/openbsd/src/blob/master/lib/libc/stdlib/r...
- derf_ 8y ago
  It depends on what operation could potentially overflow. Mozilla has a CheckedInt (C++) type that checks all operations for overflow. You can see examples of how this is actually implemented here: <https://searchfox.org/mozilla-central/source/mfbt/CheckedInt.h#254> https://searchfox.org/mozilla-central/source/mfbt/CheckedInt.... You can derive similar C implementations if you work through all of the template magic. Be warned that the definition of what is undefined, implementation-specific, and well-defined varies depending on the exact version of the C or C++ standard you use.
- raverbashing 8y ago
  Well, the way you test are usually: 1) you know the ranges your params belong to and 2) you avoid situations where you might overflow. Usually if you catch the overflow you already lost. I'm surprised to see that for unsigned ints modular semantics are guaranteed by the spec "Please tell me again how unsigned ints are broken" It seems this meme seems to be the C version of anti-vaxxers.
blue_pancake 8y ago
Undefined behavior gets a bad rap, but it's not always evil. Compilers and executables would be a lot slower if they had to account for these cases. If you're writing serious C, you should be using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior. The tools are there. It's just not something a lot of people do.
- nkurz 8y ago
  My instinct is to downvote this to keep it at the bottom of the page, but since you are a new user with seemingly good intentions, that seems too rude. Undefined behavior gets a bad rap, but it's not always evil. Probably true, but make sure that you are distinguishing between undefined and implementation defined. Compilers and executables would be a lot slower if they had to account for these cases. Maybe, but I'd like to see better quantification for "a lot". My instinct (unsourced) is that it's usually minimal for most C, and quite significant for templated C++. But I'd interested in seeing firm numbers on this. If you're writing serious C, you should be using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior. This is where I think you veer off into being mostly wrong. I grew up with Valgrind, but I don't think it really has much use any more. You are almost always better off with one of the now built-in "sanitizers": https://github.com/google/sanitizers/wiki/AddressSanitizerComparisonOfMemoryTools https://github.com/google/sanitizers/wiki/AddressSanitizerCo.... But while you should be using these to catch bugs, neither the sanitizers nor Valgrind are able to catch the dangerous forms of undefined behavior shown in the examples in the article. UBSan is great and should be more used than it is (https://medium.com/@lucianoalmeida1/the-undefined-behavior-sanitizer-6e7fe78790c7 https://medium.com/@lucianoalmeida1/the-undefined-behavior-s...) but it is not going to catch anything near all the problems with undefined behavior! Here's a summary of the unfortunate state of the art: https://blog.regehr.org/archives/1520 https://blog.regehr.org/archives/1520. While there are underutilized tools that can help, "using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior" is likely to give you misplaced confidence that you are free from the dangers.
  - twtw 8y ago
nayuki 8y ago
I agree with everything in the article - the example of non-intuitive effects of the strict aliasing rule, a tricky integer overflow example, and the unpopular plea to switch away from C/C++. When I write C and C++ code, I try to make my logic portable and standards-compliant so that it will work on all platforms. So instead of assuming int is 32 bits, I am only allowed to assume that int is at least 16 bits wide. I assume that sizeof(char) could equal sizeof(int) and both could be 64 bits. I avoid bitwise manipulation on negative numbers, because they're not guaranteed to be two's complement. Keeping all of these pessimistic assumptions in mind while I code is a mental burden that I don't experience in other languages. Regarding integer promotions, here is one tricky situation I reasoned about and asked in https://stackoverflow.com/questions/39964651/is-masking-before-unsigned-left-shift-in-c-c-too-paranoid https://stackoverflow.com/questions/39964651/is-masking-befo... . Suppose you want to compute: uint32_t a = UINT32_C(0xFFFFFFFF); uint32_t b = a << 31; b should be 0x80000000 Looks innocent, eh? Left-shifting an unsigned integer will discard the top bits and never cause undefined behavior. Except, this reasoning can be wrong on some platforms. Suppose: typedef unsigned short uint32_t; typedef int int48_t; Now (uint32_t)a → (unsigned short)a → (int)a → (int48_t)a, due to typedefs and integer promotion. But because a is a signed integer, it is undefined behavior to shift 1's into the sign bit. Kaboom.
- jjnoakes 8y ago
  I wonder if a future c standard can fix this in some way. Say by promoting unsigned types to unsigned int instead of int...
- garaetjjte 8y ago
  >I avoid bitwise manipulation on negative numbers, because they're not guaranteed to be two's complement. This seems purely theoretical, as there is no reasonable architectures that are not using two's complement. It will be defined as two's complement in C++20. (but still with signed overflow UB, to keep some optimizations possible)
- baby 8y ago
  I try to never use int personally, if uint8_t et al. exist, it's for a reason.
jancsika 8y ago
> The C standard specifies that values “cannot” be accessed through pointers that do not match the effective type of the value Yet they can be and often are by using the union trick. To decide whether to use the union trick requires a discussion of a program's desired portability which-- while usually desirable-- is a separate issue from undefined behavior.
- FartyMcFarter 8y ago
  On top of this, unions behave differently in C++ wrt undefined behaviour.
- mort96 8y ago
  The union thing is still 100% undefined behavior though, right? Some specific compilers guarantee a particular behavior, but by relying on it, you're not _really_ writing standard C anymore.
  - vyodaiken 8y ago
    The rule allows both the union trick and a totally hacky char type exception (this was added because the standards authors found out that their rule was inconsistent and instead of backing it out, tried to hack it up). 7 An object shall have its stored value accessed only by an lvalue expression that has one of the following types:88) — a type compatible with the effective type of the object, — a qualified version of a type compatible with the effective type of the object, — a type that is the signed or unsigned type corresponding to the effective type of the object, — a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object, — an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or — a character type
sn41 8y ago
> The C standard specifies that values “cannot” be accessed through pointers that do not match the effective type of the value I think this is used in "type punning" in union structures. This is a related comment by Linus Torvalds on the kernel list: https://lkml.org/lkml/2018/6/5/769 https://lkml.org/lkml/2018/6/5/769
liftbigweights 8y ago
Actually undefined behavior is defined. It is defined as undefined.
- yjftsjthsd-h 8y ago
  ...and thus is undefined. In what possible sense is there a useful distinction between "accidentally" undefined and "intentionally" undefined?
twtw 8y ago
Why not just define these things? Make -fwrapv, -fno-strict-aliasing the standard?
- gpderetta 8y ago
  Because many (most?) Don't want them.
  - twtw 8y ago
    Any specific examples? I know Linux, at least, uses these flags to compile, and Linus scorns these parts of the standard.
    - vyodaiken 8y ago
      There is no practical use for those UB "optimizations".
- garaetjjte 8y ago
  Because it prevents some optimizations, eg. there is one reason for signed overflow UB: https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759de5a7 https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759...
- vinkelhake 8y ago
  One argument (that is unrelated to optimization) for keeping signed wrap undefined is that most instances of it is a real bug. By keep signed wrap undefined, you now have the option of using -ftrapv to weed out such bugs.
  - jforberg 8y ago
    That might be a valid argument for making -ftrapv the standard. It's certainly not an argument for keeping it undefined. Most -f options tweak the standard in some way and defining overflow would not make -ftrapv suddenly go away.
ajnin 8y ago
Is there a compiler flag that can be set to print a warning about all undefined behaviour? I'm no a C dev but this UB business seems like a little cat and mouse game between the developer and the compiler which tries to find "tricks" to avoid doing stuff, which seems backwards.
- gpderetta 8y ago
  In the vast majority of the cases, undefined behaviour is impossible to detect a compile time.
- aw1621107 8y ago
  Chris Lattner addresses this in part 3 of his “What Every C Programmer Should Know About Undefined Behavior” [0]: “People often ask why the compiler doesn't produce warnings when it is taking advantage of undefined behavior to do an optimization, since any such case might actually be a bug in the user code. The challenges with this approach are that it is 1) likely to generate far too many warnings to be useful - because these optimizations kick in all the time when there is no bug, 2) it is really tricky to generate these warnings only when people want them, and 3) we have no good way to express (to the user) how a series of optimizations combined to expose the opportunity being optimized.” Granted, this was written in 2011, so there’s a chance that something has changed, but the points brought up there seem like it isn’t that easy to fix. [0]: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_21.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
- yjftsjthsd-h 8y ago
  I suspect such an option would be so verbose as to be useless; I believe it is virtually impossible to write any substantial amount of C without encountering undefined-behavior. It still might have some value for writing programs if you're willing to put in the absurd amount of effort required to actually be safe, but now we're getting into the territory of OpenBSD or qmail; possible, but vanishingly rare.
- zielmicha 8y ago
  Address/Undefined Sanitizer (-fsanitize=undefined) will detect many undefined behaviours (but at runtime).
vyodaiken 8y ago
Just a reminder that there are ZERO published studies showing that these UB "optimizations" have significant value for any real programs. They impose a bizarre notion of C semantics that is not compatible with the language design. A good critique, for example, of the UB alias behavior can be found in Brian Kernhighans article on Pascal ( http://www.cs.virginia.edu/~evans/cs655-S00/readings/bwk-on-pascal.html http://www.cs.virginia.edu/~evans/cs655-S00/readings/bwk-on-... ). The Standards authors have made the exact same error but in an ad hoc hacked up manner. Just use the flags that Linux has forced on the compiler developers in order to be able to make use of C or else give up on writing correct code. http://www.yodaiken.com/2018/11/17/standard-c-is-more-fun-than-ordinary-c/ http://www.yodaiken.com/2018/11/17/standard-c-is-more-fun-th...
- wolf550e 8y ago
  code patterns like this happen a lot and the optimization is worthwhile: https://www.youtube.com/watch?v=yG1OZ69H_-o&t=2359 https://www.youtube.com/watch?v=yG1OZ69H_-o&t=2359
  - vyodaiken 8y ago
    Again: not a single study with a benchmark, just an anecdote that actually means the opposite of what is intended. The authors of Bzip had to write non-idiomatic code to prevent UB "optimization" from breaking their code, and this results in compiled code that may or may not be slower. Tip: fewer assembly instructions does not always mean faster execution in a pipelined processor. In fact, compiling with -fwrap probably would generate better code.
    - vinkelhake 8y ago
      Do you have an example (or maybe a published study) where -fwrapv generates better code?
      - vyodaiken 8y ago
        Do you have a single published study showing that the UB "optimizations" benefit real programs? I typed in the example code and current Clang generates exactly the same assembly for both u and not u integers - which is correct. So the problem he is describing is an error in Clang code generation that has since been fixed. Again: I have yet to see a serious attempt to validate the utility of UB "optimizations" - this video certainly does not contain any.
AndyKelley 8y ago
In Zig we embrace undefined behavior. It's what allows tools to catch mistakes, and it's what arms the optimizer with the assumptions it needs to be effective. For example, not only is it undefined behavior to overflow on signed integer addition, in Zig it's also undefined behavior to overflow on unsigned integer addition. If you want wrapping integer addition, you have to use the wrapping integer addition operator, which is defined to wraparound on overflow. Here's the trick though - Zig catches most kinds of undefined behavior before they have a chance to cause problems in release builds. Some undefined behavior is caught at compile time, and otherwise most undefined behavior is caught at runtime, in debug builds. And finally, if you are not confident in the level of testing your software has undergone, you can make a "release-safe" build, which has optimizations on, but includes undefined behavior checks and will crash (or invoke user-defined panic function) rather than invoke undefined behavior. You can see some examples of this here: https://ziglang.org/documentation/master/#Undefined-Behavior https://ziglang.org/documentation/master/#Undefined-Behavior
- Filligree 8y ago
  If the programmer can rely on your behavior, i.e. its being caught, then... that's not really undefined behavior, then. The entire point of undefined behavior is that anything can happen. Wrap-around, segmentation faults, elided security checks or even ravenous polar bears, anything goes, and you can't know in advance what will.
  - AndyKelley 8y ago
    It's undefined behavior when you compile in release-fast or release-small modes.
tempodox 8y ago
Articles like this are important. The superficial simplicity of C can be misleading.
eridius 8y ago
The penultimate example tripped me up. I was under the impression that arithmetic conversions for binary operators only happened if the types of the operands were different. But reading the standard, and then actually experimenting with clang and __auto_type, does confirm that if the operands can be converted to int or unsigned int, then they will be (and that it will convert to int if int can represent all the values). That's really kind of nasty given the lack of wrapping overflow on signed integers. This actually makes me wonder, if I do want wrapping overflow on signed integers in C, how do I request it? Is there some compiler builtin or stdlib function to say "please add/multiply/whatever these signed integers with overflow"?
- jforberg 8y ago
  You have several options, at least if you are willing to stick with GCC/Clang. Passing -fwrapv will give you well-defined (wrapping) signed overflow, which is often the native machine behaviour anyway. If you are interested in catching overflow when it happens, you can use intrinsics like __builtin_add_overflow. These look at the overflow flags present on many machines to let you know if the operation overflowed so you can handle it any way you like. If you are of the opinion that overflow should never happen in your program, you can pass -ftrapv which asks the compiler to crash your program if overflow ever occurs.
ThisIs_MyName 8y ago
More of this: https://www.youtube.com/watch?v=yG1OZ69H_-o https://www.youtube.com/watch?v=yG1OZ69H_-o

94 points by erwan 8y ago

saagarjha 8y ago

There’s really no need to pass -O9 to GCC. Anything over -O3 should become -O3 anyways.

jibal 8y ago
It doesn't hurt.
- jnurmine 8y ago
  I wonder if it does anything useful at all, since it -O9 is not defined, unless the compiler is some kind of special version.
  - yjftsjthsd-h 8y ago
    IIRC, the code is actually `if O>2` or so; 3 and up are literally identical. EDIT: Per https://wiki.gentoo.org/wiki/GCC_optimization#What_about_-O_levels_higher_than_3.3F https://wiki.gentoo.org/wiki/GCC_optimization#What_about_-O_... it's `enabled = (level >= 3);`
- johnisgood 8y ago
  It puts the article in a different light. Makes you wonder what else he is incompetent or negligent about. In this sense it does hurt: it hurts him, it makes him less credible.
  - baby 8y ago
    you realize who the author of the article is do you?
    - johnisgood 8y ago
      What are you trying to say? That because he is an authority figure to you and perhaps to other people, it is somehow OK to be ignorant about certain things? Last time I did something similar it was due to sheer ignorance and placebo. It might be a bad mistake, or a terrible habit, but it is what it is. It is silly regardless of who it comes from. :)
      - baby 8y ago
        He is competent, and he wrote a blogpost to share something he knows on the web. It's a bit goofy to start claiming things about a character you don't know from a free publication.
        johnisgood 8y ago
        Let me repeat: it is silly regardless of who it comes from. Everything else is irrelevant. You appear to be biased to the point of completely ignoring everything I have said. I get it, you like him. It does not change anything. I merely suggested the possibility that he might be negligent or wrong about other relevant things. Do you think that this is impossible?
        jibal 8y ago
        The author is competent and not at all silly. And the author explicitly stated what -O9 does and why he used it. > You appear to be biased to the point of completely ignoring everything I have said. This is in violation of you usage agreement with ycombinator. > I merely suggested the possibility that he might be negligent or wrong about other relevant things. No, that's not true. > Do you think that this is impossible? Strawman. I see that you created your account just a few days ago ... you might want to be careful with it. In any case, I won't be engaging with you further, and I doubt that I'm the only one.
        johnisgood 8y ago
        > This is in violation of you usage agreement with ycombinator. I do not see how it would be the case. > No, that's not true. Care to elaborate as to why you think that it is impossible for him to be negligent or wrong? > I see that you created your account just a few days ago What does it have to do with anything? > you might want to be careful with it Are you threatening me? > I won't be engaging with you Okay. > I doubt that I'm the only one Perhaps.
pornin 8y ago
In older times (around the GCC 2.7 series, I think), "-O9" was documented to be the "future-proof" setting that selects the maximum optimization level, whatever it is. In practice, GCC never defined anything beyond -O3, but I still use -O9 when I want to make GCC do its best (or worst). Call it force of habit from a lengthening experience. (With Clang I use -O3 because if I write -O9 it screams at me.) (Note that -O3 is usually not that good an idea: aggressive loop unrolling can sometimes decrease performance because of cache issues, for instance. To optimize code properly, you have to make benchmarks and adjust both the code and compiler flags accordingly, in a well-thought feedback cycle. The use of aggressive optimizations in the blog post is to make the examples trip UB more clearly.)

kibwen 8y ago

Upvoted for the signed integer overflow example. I'll admit that I actually don't know the most idiomatic, bulletproof way of testing for signed overflow in C; if you google "how to test signed integer overflow in C", the very first result is essentially equivalent to the buggy example in the blog post ( https://www.geeksforgeeks.org/check-for-integer-overflow/ https://www.geeksforgeeks.org/check-for-integer-overflow/ ), and I'm not keen to repeat the legendary case of signed overflow within the PHP interpreter: https://web.archive.org/web/20120412194929/http://use.perl.org/use.perl.org/_Aristotle/journal/33448.html https://web.archive.org/web/20120412194929/http://use.perl.o...

0xcde4c3db 8y ago
I'm not sure there is a reliable, generic approach for detecting overflow in conformant C; you probably have to come up with the appropriate proof for the specific operation that you're about to do. OpenBSD does it for multiplication in reallocarray() [1], which was introduced because it was easy to code an integer overflow when doing something like malloc(sizeof(member) * nmembers). [1] https://github.com/openbsd/src/blob/master/lib/libc/stdlib/reallocarray.c https://github.com/openbsd/src/blob/master/lib/libc/stdlib/r...
derf_ 8y ago
It depends on what operation could potentially overflow. Mozilla has a CheckedInt (C++) type that checks all operations for overflow. You can see examples of how this is actually implemented here: <https://searchfox.org/mozilla-central/source/mfbt/CheckedInt.h#254> https://searchfox.org/mozilla-central/source/mfbt/CheckedInt.... You can derive similar C implementations if you work through all of the template magic. Be warned that the definition of what is undefined, implementation-specific, and well-defined varies depending on the exact version of the C or C++ standard you use.
raverbashing 8y ago
Well, the way you test are usually: 1) you know the ranges your params belong to and 2) you avoid situations where you might overflow. Usually if you catch the overflow you already lost. I'm surprised to see that for unsigned ints modular semantics are guaranteed by the spec "Please tell me again how unsigned ints are broken" It seems this meme seems to be the C version of anti-vaxxers.

blue_pancake 8y ago

Undefined behavior gets a bad rap, but it's not always evil. Compilers and executables would be a lot slower if they had to account for these cases. If you're writing serious C, you should be using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior. The tools are there. It's just not something a lot of people do.

nkurz 8y ago
My instinct is to downvote this to keep it at the bottom of the page, but since you are a new user with seemingly good intentions, that seems too rude. Undefined behavior gets a bad rap, but it's not always evil. Probably true, but make sure that you are distinguishing between undefined and implementation defined. Compilers and executables would be a lot slower if they had to account for these cases. Maybe, but I'd like to see better quantification for "a lot". My instinct (unsourced) is that it's usually minimal for most C, and quite significant for templated C++. But I'd interested in seeing firm numbers on this. If you're writing serious C, you should be using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior. This is where I think you veer off into being mostly wrong. I grew up with Valgrind, but I don't think it really has much use any more. You are almost always better off with one of the now built-in "sanitizers": https://github.com/google/sanitizers/wiki/AddressSanitizerComparisonOfMemoryTools https://github.com/google/sanitizers/wiki/AddressSanitizerCo.... But while you should be using these to catch bugs, neither the sanitizers nor Valgrind are able to catch the dangerous forms of undefined behavior shown in the examples in the article. UBSan is great and should be more used than it is (https://medium.com/@lucianoalmeida1/the-undefined-behavior-sanitizer-6e7fe78790c7 https://medium.com/@lucianoalmeida1/the-undefined-behavior-s...) but it is not going to catch anything near all the problems with undefined behavior! Here's a summary of the unfortunate state of the art: https://blog.regehr.org/archives/1520 https://blog.regehr.org/archives/1520. While there are underutilized tools that can help, "using tools like valgrind on debug-mode executables to make sure you aren't relying on undefined behavior" is likely to give you misplaced confidence that you are free from the dangers.
- twtw 8y ago

nayuki 8y ago

I agree with everything in the article - the example of non-intuitive effects of the strict aliasing rule, a tricky integer overflow example, and the unpopular plea to switch away from C/C++. When I write C and C++ code, I try to make my logic portable and standards-compliant so that it will work on all platforms. So instead of assuming int is 32 bits, I am only allowed to assume that int is at least 16 bits wide. I assume that sizeof(char) could equal sizeof(int) and both could be 64 bits. I avoid bitwise manipulation on negative numbers, because they're not guaranteed to be two's complement. Keeping all of these pessimistic assumptions in mind while I code is a mental burden that I don't experience in other languages. Regarding integer promotions, here is one tricky situation I reasoned about and asked in https://stackoverflow.com/questions/39964651/is-masking-before-unsigned-left-shift-in-c-c-too-paranoid https://stackoverflow.com/questions/39964651/is-masking-befo... . Suppose you want to compute: uint32_t a = UINT32_C(0xFFFFFFFF); uint32_t b = a << 31; b should be 0x80000000 Looks innocent, eh? Left-shifting an unsigned integer will discard the top bits and never cause undefined behavior. Except, this reasoning can be wrong on some platforms. Suppose: typedef unsigned short uint32_t; typedef int int48_t; Now (uint32_t)a → (unsigned short)a → (int)a → (int48_t)a, due to typedefs and integer promotion. But because a is a signed integer, it is undefined behavior to shift 1's into the sign bit. Kaboom.

jjnoakes 8y ago
I wonder if a future c standard can fix this in some way. Say by promoting unsigned types to unsigned int instead of int...
garaetjjte 8y ago
>I avoid bitwise manipulation on negative numbers, because they're not guaranteed to be two's complement. This seems purely theoretical, as there is no reasonable architectures that are not using two's complement. It will be defined as two's complement in C++20. (but still with signed overflow UB, to keep some optimizations possible)
baby 8y ago
I try to never use int personally, if uint8_t et al. exist, it's for a reason.

jancsika 8y ago

> The C standard specifies that values “cannot” be accessed through pointers that do not match the effective type of the value Yet they can be and often are by using the union trick. To decide whether to use the union trick requires a discussion of a program's desired portability which-- while usually desirable-- is a separate issue from undefined behavior.

FartyMcFarter 8y ago
On top of this, unions behave differently in C++ wrt undefined behaviour.
mort96 8y ago
The union thing is still 100% undefined behavior though, right? Some specific compilers guarantee a particular behavior, but by relying on it, you're not _really_ writing standard C anymore.
- vyodaiken 8y ago
  The rule allows both the union trick and a totally hacky char type exception (this was added because the standards authors found out that their rule was inconsistent and instead of backing it out, tried to hack it up). 7 An object shall have its stored value accessed only by an lvalue expression that has one of the following types:88) — a type compatible with the effective type of the object, — a qualified version of a type compatible with the effective type of the object, — a type that is the signed or unsigned type corresponding to the effective type of the object, — a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object, — an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or — a character type

sn41 8y ago

> The C standard specifies that values “cannot” be accessed through pointers that do not match the effective type of the value I think this is used in "type punning" in union structures. This is a related comment by Linus Torvalds on the kernel list: https://lkml.org/lkml/2018/6/5/769 https://lkml.org/lkml/2018/6/5/769

liftbigweights 8y ago

Actually undefined behavior is defined. It is defined as undefined.

yjftsjthsd-h 8y ago
...and thus is undefined. In what possible sense is there a useful distinction between "accidentally" undefined and "intentionally" undefined?

twtw 8y ago

Why not just define these things? Make -fwrapv, -fno-strict-aliasing the standard?

gpderetta 8y ago
Because many (most?) Don't want them.
- twtw 8y ago
  Any specific examples? I know Linux, at least, uses these flags to compile, and Linus scorns these parts of the standard.
  - vyodaiken 8y ago
    There is no practical use for those UB "optimizations".
garaetjjte 8y ago
Because it prevents some optimizations, eg. there is one reason for signed overflow UB: https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759de5a7 https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759...
vinkelhake 8y ago
One argument (that is unrelated to optimization) for keeping signed wrap undefined is that most instances of it is a real bug. By keep signed wrap undefined, you now have the option of using -ftrapv to weed out such bugs.
- jforberg 8y ago
  That might be a valid argument for making -ftrapv the standard. It's certainly not an argument for keeping it undefined. Most -f options tweak the standard in some way and defining overflow would not make -ftrapv suddenly go away.

ajnin 8y ago

Is there a compiler flag that can be set to print a warning about all undefined behaviour? I'm no a C dev but this UB business seems like a little cat and mouse game between the developer and the compiler which tries to find "tricks" to avoid doing stuff, which seems backwards.

gpderetta 8y ago
In the vast majority of the cases, undefined behaviour is impossible to detect a compile time.
aw1621107 8y ago
Chris Lattner addresses this in part 3 of his “What Every C Programmer Should Know About Undefined Behavior” [0]: “People often ask why the compiler doesn't produce warnings when it is taking advantage of undefined behavior to do an optimization, since any such case might actually be a bug in the user code. The challenges with this approach are that it is 1) likely to generate far too many warnings to be useful - because these optimizations kick in all the time when there is no bug, 2) it is really tricky to generate these warnings only when people want them, and 3) we have no good way to express (to the user) how a series of optimizations combined to expose the opportunity being optimized.” Granted, this was written in 2011, so there’s a chance that something has changed, but the points brought up there seem like it isn’t that easy to fix. [0]: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_21.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
yjftsjthsd-h 8y ago
I suspect such an option would be so verbose as to be useless; I believe it is virtually impossible to write any substantial amount of C without encountering undefined-behavior. It still might have some value for writing programs if you're willing to put in the absurd amount of effort required to actually be safe, but now we're getting into the territory of OpenBSD or qmail; possible, but vanishingly rare.
zielmicha 8y ago
Address/Undefined Sanitizer (-fsanitize=undefined) will detect many undefined behaviours (but at runtime).

vyodaiken 8y ago

Just a reminder that there are ZERO published studies showing that these UB "optimizations" have significant value for any real programs. They impose a bizarre notion of C semantics that is not compatible with the language design. A good critique, for example, of the UB alias behavior can be found in Brian Kernhighans article on Pascal ( http://www.cs.virginia.edu/~evans/cs655-S00/readings/bwk-on-pascal.html http://www.cs.virginia.edu/~evans/cs655-S00/readings/bwk-on-... ). The Standards authors have made the exact same error but in an ad hoc hacked up manner. Just use the flags that Linux has forced on the compiler developers in order to be able to make use of C or else give up on writing correct code. http://www.yodaiken.com/2018/11/17/standard-c-is-more-fun-than-ordinary-c/ http://www.yodaiken.com/2018/11/17/standard-c-is-more-fun-th...

wolf550e 8y ago
code patterns like this happen a lot and the optimization is worthwhile: https://www.youtube.com/watch?v=yG1OZ69H_-o&t=2359 https://www.youtube.com/watch?v=yG1OZ69H_-o&t=2359
- vyodaiken 8y ago
  Again: not a single study with a benchmark, just an anecdote that actually means the opposite of what is intended. The authors of Bzip had to write non-idiomatic code to prevent UB "optimization" from breaking their code, and this results in compiled code that may or may not be slower. Tip: fewer assembly instructions does not always mean faster execution in a pipelined processor. In fact, compiling with -fwrap probably would generate better code.
  - vinkelhake 8y ago
    Do you have an example (or maybe a published study) where -fwrapv generates better code?
    - vyodaiken 8y ago
      Do you have a single published study showing that the UB "optimizations" benefit real programs? I typed in the example code and current Clang generates exactly the same assembly for both u and not u integers - which is correct. So the problem he is describing is an error in Clang code generation that has since been fixed. Again: I have yet to see a serious attempt to validate the utility of UB "optimizations" - this video certainly does not contain any.

AndyKelley 8y ago

In Zig we embrace undefined behavior. It's what allows tools to catch mistakes, and it's what arms the optimizer with the assumptions it needs to be effective. For example, not only is it undefined behavior to overflow on signed integer addition, in Zig it's also undefined behavior to overflow on unsigned integer addition. If you want wrapping integer addition, you have to use the wrapping integer addition operator, which is defined to wraparound on overflow. Here's the trick though - Zig catches most kinds of undefined behavior before they have a chance to cause problems in release builds. Some undefined behavior is caught at compile time, and otherwise most undefined behavior is caught at runtime, in debug builds. And finally, if you are not confident in the level of testing your software has undergone, you can make a "release-safe" build, which has optimizations on, but includes undefined behavior checks and will crash (or invoke user-defined panic function) rather than invoke undefined behavior. You can see some examples of this here: https://ziglang.org/documentation/master/#Undefined-Behavior https://ziglang.org/documentation/master/#Undefined-Behavior

Filligree 8y ago
If the programmer can rely on your behavior, i.e. its being caught, then... that's not really undefined behavior, then. The entire point of undefined behavior is that anything can happen. Wrap-around, segmentation faults, elided security checks or even ravenous polar bears, anything goes, and you can't know in advance what will.
- AndyKelley 8y ago
  It's undefined behavior when you compile in release-fast or release-small modes.

tempodox 8y ago

Articles like this are important. The superficial simplicity of C can be misleading.

eridius 8y ago

The penultimate example tripped me up. I was under the impression that arithmetic conversions for binary operators only happened if the types of the operands were different. But reading the standard, and then actually experimenting with clang and __auto_type, does confirm that if the operands can be converted to int or unsigned int, then they will be (and that it will convert to int if int can represent all the values). That's really kind of nasty given the lack of wrapping overflow on signed integers. This actually makes me wonder, if I do want wrapping overflow on signed integers in C, how do I request it? Is there some compiler builtin or stdlib function to say "please add/multiply/whatever these signed integers with overflow"?

jforberg 8y ago
You have several options, at least if you are willing to stick with GCC/Clang. Passing -fwrapv will give you well-defined (wrapping) signed overflow, which is often the native machine behaviour anyway. If you are interested in catching overflow when it happens, you can use intrinsics like __builtin_add_overflow. These look at the overflow flags present on many machines to let you know if the operation overflowed so you can handle it any way you like. If you are of the opinion that overflow should never happen in your program, you can pass -ftrapv which asks the compiler to crash your program if overflow ever occurs.

ThisIs_MyName 8y ago

More of this: https://www.youtube.com/watch?v=yG1OZ69H_-o https://www.youtube.com/watch?v=yG1OZ69H_-o