10 ms·
The Value of Undefined Behavior
- samatman 8y agoThis is a fine opportunity to link to one of my favorite essays on the subject, Undefined Intimacy With the Machine: http://thoughtmesh.net/publish/367.php http://thoughtmesh.net/publish/367.php This argues that it is precisely C's undefined behavior which allowed for its dominance.
- arcticbull 8y agoI have a hard time accepting the premise that a language is better because you have to hope that it does what you want instead of telling it, and for that reason it has become successful. In fact, the ANSI C spec even states from your link that "Undefined behavior gives the implementer license not to catch certain program errors that are difficult to diagnose." That's just not an acceptable tradeoff anymore -- in fact I don't know it ever was. That's akin to shipping your org chart because you can't make your program do what everyone agrees it should do. You know, catch programming errors, in this case.
- samatman 8y agoThe paper argues that some behaviors being undefined let compilers target the widely varying architectures of the early microcomputer era, while still using a language that remains at least somewhat portable between them.
- arcticbull 8y agoYes, but your program isn't doing what you think you told it to, which is worse, because the compiler tells you that it will, know what I mean? A false guarantee of safety is kind of the worst.
- DSMan195276 8y ago> Yes, but your program isn't doing what you think you told it to, Your program is still doing what you told it to if you weren't relying on any undefined behavior (Which, obviously, you shouldn't be). That's the point - if you write C that doesn't rely on any UB, then it can run on a large variety of architectures, work exactly the same on each one, and still get close to the maximum speed for each architecture because they can take the fastest possible option for any of the UB cases (And your program shouldn't care what they do in that situation).
- fiddlerwoaroof 8y agoThe problem I generally have with C is that the scope of UB is really wide. Languages like Common Lisp have UB too, but it’s generally easier to avoid in “normal” code.
- dfox 8y agoIn CL case the "is an error" wording generally covers all of C's "implementation-defined", "unspecified" and "undefined". And surprising amount of normal looking constructs are in the implementation-defined category. When you add type declarations and (optimize (safety 0)) into the mix you get similar scope of UB as in C (another thing is that in reality nobody actually writes code like that)
- fiddlerwoaroof 8y agoI’m thinking of things like it being UB to modify a reader literal because the reader is allowed to do things like tail sharing.
- dfox 8y agoI think that modifiing reader literal is an error mostly to allow implementations to store such pairs in read-only pages (eg. in .rodata section when you build standalone executables) and to allow simple implementation of CDR-coding for reader literals (dfsch has three distinct implementations of abstract <pair> class for exactly this reason and to be able to actually signal meaningful error condition when you violate this restriction).
- ced 8y agoThat's part of the point of Worse is Better, too. https://www.jwz.org/doc/worse-is-better.html https://www.jwz.org/doc/worse-is-better.html
- arcticbull 8y agoIts undefined because it's not guaranteed to happen, that's the whole point, and that's what makes it undesirable. Rust has it right here: don't allow undefined behavior and make the programmer specify the behavior they want. If they want wrapping overflow, fine. If they don't, great. Tell the compiler don't make it guess/assume/optimize/hope. What this article is arguing for IMO is not the concept of behavior that happens to sometimes work and sometimes not depending on the target machine (because thats crazy) but the functionality itself, which is valuable, and should be explicit and defined. The issue is the last-generation languages like C/C++ don't give you mechanisms to express what you want so you're left hoping the compiler figures out what you meant. That's not a world I want to live in anymore, we can do better. UB is bad, period. Define what you want, make the compiler do it.
- kibwen 8y agoThough note that Rust still has things that are definitely undefined behavior (my favorite: "Transmuting an & to &mut is always UB; No you can't do it; No you're not special" https://doc.rust-lang.org/beta/nomicon/transmutes.html https://doc.rust-lang.org/beta/nomicon/transmutes.html ), it's just that someone has to break out the `unsafe` keyword to get it to happen, and that an error as a result of undefined behavior can always be traced back to an `unsafe` block's invariants being violated somehow.
- arcticbull 8y agoDefinitely, but at least when you see unsafe {} you know all bets are off instead of having to assume all bets are off all the time. I do see what you mean though. The transmute thing is pretty funny imo.
- cwzwarich 8y agoSince any Rust function may use unsafe code, you don't know whether bets are off for any particular function call.
- 8y ago
- taneq 8y ago"Value" is the wrong word, because it implies goodness. It's taken me a while to come around to this viewpoint but I think "undefined behaviour" is unambiguously bad in pretty much every context. I understand why it was originally included in language specs but in modern times, where even the meanest embedded processors are generally 32 bit 2's complement processors with an ALU, we can afford to define the behaviour of the language which we're using.
- millstone 8y agoOne example of undefined behavior is dereferencing a free'd pointer. How should that be defined?
- taneq 8y agoIn a perfect world it would either be impossible by design or result in a compile error. I'd settle for "reliably crash with an error message saying where in the code it happened".
- bluecalm 8y agoYou can get it in C. Just set the pointer to NULL after free, you will get your crash, problem solved.
- taneq 8y agoTrue, you can even macro it for more convenience. I'm more worried about things where some obscure edge case causes the compiler to optimise away something important because it's deemed to be UB.
- lomnakkus 8y agoNo, you'll get UB.
- bluecalm 8y agoObviously you check for NULL first. If something may or may not exists you first ask if it exists (check for NULL) and then of it exists you do something with it. When you free something you destroy it, that's why it's useful to set the pointer which before pointed to it to NULL.
- Asooka 8y agoI really think that undefined behaviour should be switched to machine-dependent behaviour everywhere and we should just have tools that can suggest optimized variants of hot functions - to be rewritten in ways that take advantage of assuming undefined behaviour doesn't happen. Or lift the `extern "lang"{}' construct from C++ to C and have `extern "C-strict"' which turns on full strict undefined behaviour optimizations. Yes, I know I can turn most of them off. We ship C++ software and always compile with singed-overflow enabled and strict-aliasing disabled (among others). Those are just too dangerous to be turned on for an entire codebase of a large C++ application where you have no idea what will be inlined where and what LTO will do. But if some UB allows for measurable performance gains, we should be able to turn it back on for restricted safe scopes. And please, if the compiler can prove that the program invokes UB in "strict-UB" mode, that should be an error, not a cause for the code to be deleted. I'm talking about the case of this SPEC 2006 code [1] It doesn't make sense to me to just apply potentially disastrous micro-optimizations that save a couple of percent of run-time on the whole program. This sounds like a classic case of optimizing before measuring. 1 https://blog.regehr.org/archives/918 https://blog.regehr.org/archives/918
- tedunangst 8y ago> This sounds like a classic case of optimizing before measuring. But you have done the measurements, right?
- gpderetta 8y agoHow would you make machine defined the behaviour of, say, use after free, or data races? Also compilers normally do not prove that programs hit UB, quite the contrary. Doing that statically is, in general, impossible.
- IshKebab 8y agoThis is a good history lesson on why UB was introduced, but times have changed and it is definitely not justifiable today. Saving one instruction was probably worth it in the 70s, when security wasn't a thing, but it definitely isn't now.
- iainmerrick 8y agoIf these are the best justifications we can come up with, it doesn't change my opinion that the current reliance on UB by compilers is a bad idea. Anyone got any better ones?
- saagarjha 8y agoIt makes your code many times faster in a lot of cases?
- iainmerrick 8y agoThis article gives two examples where UB arguably helps you, and in each case the article itself mentions a cleaner and better way of doing it. Magic behavior of "int" (but not "unsigned int") --> just use a type that's the correct size and better conveys your intention, like "size_t". (And that one doesn't make your code "many times faster", in the example given it saves a single instruction.) Type-based assumptions about aliasing (with a special case for char*) --> be explicit about your intention with the "restrict" keyword.
- mehrdadn 8y agoTo those annoyed about, say, signed integer overflow being undefined: do you even have a habit of ensuring that what's inside your loop will behave correctly around an overflow boundary? Is the real problem really only the loop increment for you?
- 3pt14159 8y agoThe problem is tractability. If you send me your data, your source, and your OS and if I compile it with a slightly different compiler and I can't reproduce it gets incredibly frustrating to hunt down why something isn't working.
- mehrdadn 8y agoInteresting, this is a less common complaint than what I usually hear. It kind of makes sense if you run into this a lot (I don't in C++, but maybe it happens more in C?), but at the same time I'm wondering why can't you just e.g. use -fwrapv if you want to rely on integer wrapping in your release executable? It also doesn't sound like it's related to using a different compiler, since using the same compiler with a flag like -fwrapv or with optimizations disabled should also prevent UB from being exploited.
- 3pt14159 8y agoSorry, I miscommunicated. I was not speaking about integer wrapping, I was speaking about undefined behaviour in a general sense.
- nwmcsween 8y agoWell if the compiler wraps signed numbers implicitly you have other issues.
- makecheck 8y agoExcessive flexibility for optimizers, schedulers, etc. adds a high price to debugging, and it’s not like we have perfect programs that can only be improved in these fancier ways. Programs are not all written by experts. Performance may be left on the table in areas that have nothing to do with optimizers or schedulers (e.g. picking an absolutely terrible algorithm or not noticing unnecessary memory allocations). If something must remain “unspecified”, I at least want a debugging tool for that behavior. If something “may” happen, give me a switch to make it happen. If something is data-dependent, show me the data that triggers a different outcome or give me a way to scramble data enough to increase the chance of triggering a change in behavior.
- swayvil 8y agoDefined behavior is a mere shadow of undefined behavior. All form comes from the constraining definition. One who limits his explorations to the defined has crawled into a hole and pulled in the hole after him. Ya, I know, offtopic.
- deleted 8y ago[deleted]
- saagarjha 8y ago> What irritates a lot of people is that compilers will still apply the strict aliasing rule even when it’s trivial for the compiler to prove that aliasing is occurring I think better diagnostics for this would be a win for everyone.
- MrBingley 8y agoThe problem with the first example is that `int` and `unsigned` are not the correct types to be using for array indexing. As we saw, they were 32 bits wide on a 64 bit platform, which lead to an additional sign extension and (in the case of `unsigned`) truncation instruction. However, this can be avoided by using `ptrdiff_t` and `size_t`, which are defined to match the platform size and avoid the extension and truncation altogether. The fact that signed integer overflow being undefined allowed the compiler to elide the `int` truncation is somewhat irrelevant, since using the correct integer types leads to even faster code that doesn't rely on undefined behaviour at all.
- Dylan16807 8y ago* Most forms of undefined behavior don't have any meaningful use to compilers. * With integer overflow it would be relatively easy to write a spec that allows "n+1>n" optimizations but not utterly anything to happen. With strict aliasing it's probably not too hard either. You'd still see some program misbehavior in the strict aliasing case, but it would be a vastly reduced set of misbehavior. And allowing signed numbers to act more like abstract numbers would often decrease misbehavior.