10 ms·
Undefined behavior is often a thing for C programmers (2011)
- flohofwoe 6y agoUnpopular opinion: the problem isn't so much undefined behaviour, but rather that compiler vendors have chosen to exploit undefined behaviour in their optimization passes. PS: some pragmatic cleanup of UB and IB in the C standard would be welcome though.
- chrisseaton 6y agoThis is the mainstream opinion.
- raphlinus 6y agoThis is not an unpopular opinion per se, but one that puts you firmly in the "semi-portable C" camp (in the "camp" model of the essay I linked in another comment), while the compiler vendors themselves have moved entirely away from that camp into the "standard C" camp. As I wrote, "The principles and practices in one camp can seem alien, even threatening to another."
- mastax 6y agoHow would you modify the standard and/or optimization passes for the first example in the article?
- microcow 6y ago@mastax, you could model pointer dereferencing as an effect and require dereferencing null pointers to trap.
- k1t 6y agoYou can't remove A (redundant given B) and then subsequently remove B (redundant) as well. As soon as your first pass decides to rely on the presence of the dereference as a reason for removing the null check, the dereference needs to be flagged as critical, such that it cannot be removed by a subsequent pass. Better still, if the compiler does try to remove something that was relied on by an earlier optimization step, alert the user that something is probably wrong with the code.
- MereInterest 6y agoMaybe this is me coming from a C++ background rather than C, but that sounds like a horrible idea. That would mean that any code removed by a branch determination (e.g. if(0) in a macro/template expression) would be assumed to still exist for the further purpose of removing unused variables. Also, this would prevent one of the main benefits of function inlining. Once you have inlined a function, the values of some of the parameters may be known at compile time, resulting in lots of those if(0) parameters. But that is something that was relied on in previous passes, even though it is known to be a static value in the current pass.
- deleted 6y ago[deleted]
- dooglius 6y agoWhy would it mean that? The compiler can still remove anything inside an if(0).
- shultays 6y agoThe reason for having UB in the first place is so compilers can exploit them so we can have more optimized programs. UB is a feature, not a bug. So you don't really "clean" it up. Cleaning up of UB means a lot of overhead for programs.
- geofft 6y agoYes, but also, UB is a "feature" to work around the fact that C is a relatively un-expressive language with a simple type system. It's great that the compiler can optimize for (i = 0; i < *len; i++) array[i] = 0; without having to check on each iteration whether len secretly points inside the array and got modified by the loop. It would be better if this were not a heuristic and there were clear compile-time information saying for sure that this optimization is safe.
- sharpneli 6y agoIf len and array are both of the same type then the compiler must assume they can alias (or one of them is char*). One has to explicitly state with restrict that they cannot alias. In case of similar but where it’s float array[N] etc then compiler can assume that they do not alias.
- geofft 6y agoRight, that's a heuristic, and the consequence of that heuristic is that a bunch of things where you intentionally have pointers of different types to the same memory - the sort of low-level data manipulation that C ought to be good for - is UB unless you are careful about it and you find one of the exceptions to this rule.
- sharpneli 6y agoNot a heuristic. Well defined, even if it could be defined better. I'd personally make all types not alias by default and then have a separate keyword 'alias' to specify things that must alias. One of the reasons why Fortran was faster for decades was the default aliasing rules, restrict finally got us out of that trap. If we'd always assume everything alias plenty of code would become way slower. Also C has well defined ways to access memory via different types. Make an union type and access via that. It makes it explicit.
- kps 6y agoThe problem is that the initial concept was botched by accident (C89) and then embraced rather than corrected by later committees. Nobody during C89 formulation realized that it would end up “a license for the compiler to undertake aggressive optimizations that are completely legal by the committee's rules, but make hash of apparently safe programs” that “negates every brave promise X3J11 ever made about codifying existing practices, preserving the existing body of code, and keeping (dare I say it?) `the spirit of C.'” If they had, the response would have been that “[UB] must go. This is non-negotiable.”
- MereInterest 6y agoYou have many quotation marks, but no reference to who said them. Those aren't quotes from the articles, so I can't find where they came from.
- kps 6y agoA letter to X3J11 from some chap named Dennis Ritchie.
- raphlinus 6y agoFor people who read the comments here but not the primary sources[0], you've put words into Dennis Ritchie's mouth. He did not say that UB must go, but that noalias must go. And, lo and behold, it did. So these (altered) quotes are not at all a fair characterization of what's actually in the standard. [0]: https://www.lysator.liu.se/c/dmr-on-noalias.html https://www.lysator.liu.se/c/dmr-on-noalias.html
- kps 6y agoHe said why ‘noalias must go’, and those reasons apply equally to the modern conception of UB, so it's clear that the consequences of the definition were not realized at the time.
- sharpneli 6y agoWith a nonoptimizing compiler the first example just segfaults at dereferencing P. I’m not quite sure what the compiler is supposed to do in that case. Insert nullchecks by itself to avoid crashes?
- btrask 6y agoFun fact: if there were no C language definition, there would be no "undefined behavior."
- steerablesafe 6y agoIn that case all C program behavior would be literally undefined.
- btrask 6y agoIs that what you think of Python? How can this be: C has a standard, when many other languages do not. Thus C is "more defined" than other languages. Yet somehow it ends up being less defined. If C had no standard, it would be defined by its implementation, just like many other languages. But because it has a standard, it has undefined behavior. The real trick is that without a standard, implementations have to be bug-compatible. They're too defined! The purpose of a standard is to define them less.
- steerablesafe 6y agoPython many times had the problem of being defined as CPython. Modules that target multiple Python implementations often have to work around differences between them, without a standard they have to look into implementation details of their targets. Not having a standard is definitely a pain. But for most intents and purposes Python is defined by its documentation, and it's fine. Having said that UB in C and C++ means behavior that the standard doesn't define. If you remove the standard then UB either loses meaning or it makes all programs undefined. It definitely doesn't make all programs defined.
- btrask 6y agoIf you remove the standard, wouldn't everything naturally be implementation-defined? Like, say, Python.
- 6y ago
- pornel 6y agoCompilers use UB for optimization, because programmers de-facto demand it. You'd hate a compiler where this has silly overhead: for(int i=0; i < n; i++) arr[i]; That's because 64-bit addressing modes don't overflow the same way as signed 32-bit ints. But signed overflow being UB gives compilers license to ignore the difference and generate code that makes sense. The real problem is that C is too bare-bones and lax to be able to contain or avoid UB in day to day programs. For the aforementioned overflow gotcha, there's no way to enforce use of size_t for indexing. There's no `checked_mul(a,b)` to easily avoid signed UB. You have to make the check yourself, and if you're not careful, that check will be UB too. malloc() makes you multiply numbers often, exactly where it's dangerous. And on top of that implicit integer conversions can make unsigned arithmetic surprisingly signed.
- dooglius 6y agoPresumably you meant to do something with arr[i] or the whole loop could be elided. I'm not sure I understand your point, what is the type of n? If it's something bigger than int that's a bug and should result in a warning.
- aw1621107 6y agoFrom what I understand, the type of n isn't really the important part here; it's the type of the indexing variable (i) that makes a difference. On platforms where sizeof(int) is not the size of a register (e.g., x86_64 with 32-bit ints), UB signed overflow allows the compiler to not have to emit code to deal with the possibility of 32-bit overflow, which probably results in "sensible" assembly. This article [0] goes into a bit more detail. [0]: https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759de5a7 https://gist.github.com/rygorous/e0f055bfb74e3d5f0af20690759...
- dooglius 6y agoThe article you linked says that compilers can prove a lack of overflow in this example, so UB is not necessary to assume here. It also claims that the compiler cannot prove this in more complex examples (but gives no explicit examples). What I'm saying is if the compiler cannot prove a lack of overflow in this way, that should be a warning.
- raphlinus 6y agoAs good a time as any to pimp my own essay on the topic, from a couple years ago: https://raphlinus.github.io/programming/rust/2018/08/17/undefined-behavior.html https://raphlinus.github.io/programming/rust/2018/08/17/unde...
- fortran77 6y agoAnd yet, they managed to write the Linux kernel in it, as well as thousands of other things we use every day.
- tinus_hn 6y agoObviously the most dangerous form of undefined behavior is not that which always crashes but that which for some unlikely inputs cases crashes.
- staticassertion 6y agoI'm not sure what your point is. The Linux kernel is riddled with undefined behavior and billions are spent on finding, removing, or mitigating that.
- me_me_me 6y agoI wonder what would happen if i put hand into fire. But I have never done it in my life, because it might cause 'undefined behaviour'. In my years of working with C its always the 'look how smart I am code' causes issues. Keep it clean and simple. Avoid complexity and fluff features of C. And you will not going to have deal with exotic bugs.
- merricksb 6y agoIf interested, see discussion at time of publication in 2011: https://news.ycombinator.com/item?id=2548410 https://news.ycombinator.com/item?id=2548410
- dboreham 6y agoWhy oh why does "undefined behavior in C" come up constantly here? I wrote C every day for decades. I was aware there's such a thing as undefined behavior. I can't remember it ever being even a miniscule factor in my daily work. What's changed?
- codys 6y agoOne piece of this is probably that compilers competing to generate the best code have increasingly incorporated understanding of undefined behavior into their optimizers, leading to things that were previously undefined but happened to behave in some particular way to instead behave in ways that break programs using undefined behavior. Compilers incorporate inferences that can be made by assuming UB doesn't occur into their optimization because it improves code generation of correct programs.
- ryandrake 6y agoThis is one of the humps that C programmers eventually just need to get past: That UB means literally anything is possible[1]. Not just what you currently see happening with the current version of your compiler on your current platform. Anything! We've all run into a few programmers who would insist that their particular invocation of undefined behavior was fine, because as you said, it "behaves in some particular way". Now that we have more aggressively optimizing compilers we don't run into those folks so much anymore. I wouldn't say it's a minuscule factor in my daily work, as OP put it, but it's not something to agonize about if you stay mindful of avoiding common undefined behavior pitfalls. It would be very helpful if compilers were better about warning you when you're venturing into UB, but that's a different topic. 1: You can even get the shirt: https://teespring.com/shop/undefined-behavior-shirt https://teespring.com/shop/undefined-behavior-shirt
- perl4ever 6y ago>That UB means literally anything is possible Many people say this but nobody "literally" believes it. It's a way of expressing the righteous dominance of one person or group's attitude towards reasonableness over another's. In a way that forestalls argument. "Literally anything" could literally mean global thermonuclear war.
- mijoharas 6y agothe links to part 1[0] and part 3[2] in the text are dead (at least for me). Here they are to save other people time: [0] https://blog.llvm.org/posts/2011-05-13-what-every-c-programmer-should-know/ https://blog.llvm.org/posts/2011-05-13-what-every-c-programm... [1] https://blog.llvm.org/posts/2011-05-14-what-every-c-programmer-should-know_14/ https://blog.llvm.org/posts/2011-05-14-what-every-c-programm... [2] https://blog.llvm.org/posts/2011-05-21-what-every-c-programmer-should-know_21/ https://blog.llvm.org/posts/2011-05-21-what-every-c-programm...
- not2b 6y agoThe example case is flagged by a number of static checkers, for example Coverity, as a likely error (pointer is dereferenced before it is tested for null).
- xg15 6y agoWhy the title change? The original title "Why undefined behavior is often a scary and terrible thing for C programmers" may be a bit polemic but occurs verbatim in the article - and IMO does a better job of expressing the article's message.