15 ms·
Why undefined behavior may call a never-called function
- tyingq 9y agoInteresting choice of code for a demo. I wonder if anyone hosed themselves running this. It seems gnu rm has "-preserve-root" as a default, but that's not guaranteed to be on every rm.
- csomar 9y agoOne of the advantages of using Docker! You can safely run this code multiple times and play with it without affecting your machine.
- tyingq 9y agoSure, but you could also replace "rm -rf /" with "id" or something else non destructive. Somebody is going to copy paste that snippet and have a bad day.
- dmix 9y agoI believe "rm -rf /" illustrates the point quite well, better than id. I'm more in the common sense is more common than people give credit and in the situations where that is not the case then rm has built-in protections. It literally says: > That is, the compiled program executes “rm -rf /” The next line following the code. It's not tricking anyone.
- tyingq 9y agoGnu rm has built in protections. Not sure that's the case on OSX, Linux distributions that use busybox, etc.
- jwilk 9y agoI usually use "cowsay" for demonstrating that arbitrary code execution is possible. Less destructive, and more entertaining when someone actually tries to run the code. :-)
- rcthompson 9y agoIf someone blindly copies, pastes, compiles, and runs a random snippet of code from the internet that is specifically described as producing weird and unexpected behavior, then they deserve the harsh lesson they're about to learn.
- andrewflnr 9y agoJust because it's a harsh, cruel world doesn't mean you should be cruel.
- nurettin 9y agoIt doesn't mean he should be cruel, it just means he is statistically more likely to be cruel.
- rcthompson 9y agoI normally have some measure of sympathy for people who blindly run code that is unexplained or under-explained (e.g. the many "curl | bash" installers). But blindly running code that is specifically called out as having weird and unexpected behavior is like driving your car right through the bright yellow "bridge out" sign that's blocking the bridge.
- nv-vn 9y agoDo you use Docker to run literally every piece of code though?
- ramshorns 9y agoIf it's run as an unprivileged user rather than root, it won't be able to delete the whole system. I'm pretty sure that means it won't do anything, but I don't feel like trying it.
- tyingq 9y ago>I'm pretty sure that means it won't do anything Assuming an rm without the "-preserve-root" default, it would remove everything that it had permission to. So, eventually, for example, it would wipe your home directory. I suspect this to be the case for OSX, Alpine Linux (or other distros that use busybox), probably some of the BSD distributions, etc.
- jwilk 9y ago$ uname -sr FreeBSD 11.1-RELEASE $ rm -rf / rm: "/" may not be removed No idea about other BSDs. Busybox's rm is indeed happy to nuke /.
- jwilk 9y agoAlso: $ uname -sr OpenBSD 6.1 $ rm -rf / rm: "/" may not be removed
- jwilk 9y agoThe latest POSIX standard actually requires this behavior: If […] an operand resolves to the root directory, rm shall write a diagnostic message to standard error and do nothing more with such operands. Source: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm.html http://pubs.opengroup.org/onlinepubs/9699919799/utilities/rm...
- deleted 9y ago[deleted]
- kazinator 9y agobut that's not guaranteed to be on every rm. Not even the GNU Coreutils one, if compiled with this compiler.
- beatbrokedown 9y agoIf you make one small change to this file you can cause clang and gcc to both prevent this from compiling if you are using warnings. namespace { void NeverCalled() { Do = EraseAll; } } or marking NeverCalled as static itself. Results in warning: unused function NeverCalled. In general this is best practice for functions defined and used in a single translation unit.
- saalweachter 9y agoThat's a vital part of the setup. You could hypothetically link this compilation unit against another unit which included: void NeverCalled(); struct A { A() { NeverCalled(); } }; A a;
- Annella 9y agoVery interesting. Thank you for sharing this!
- XCSme 9y agoVery interesting, to me seems like a "compiler bug". The compiler should not automatically set the static pointer value if the function that sets it is never called. Anyway, I guess "undefined behavior" is really undefined and it means anything can happen, so as per specs it's not a bug. Ultimately it's the programmer's mistake for having undefined behavior in his code.
- electrograv 9y agoUnfortunately, "it's not a bug, it's a feature!" -- there are long-standing design choices in C/C++ where various circumstances are explicitly designed to yield "undefined" behavior where literally anything goes. I believe the original intent of these are to give the compiler/optimizer more room to speed up the executable. Edit: The 'undefined' clause here is due to invoking a function at address 0, rather than any lack of variable initialization (since global variables are automatically initialized to 0, as the poster below points out).
- mikeash 9y agoGlobals are always initialized. If no initial value is specified, they’re initialized to zero. What’s undefined here is calling a function pointer that contains zero. Thus the compiler assumes that it must have been set before being called, and since there’s only one value it could possibly be set to, it must be that value.
- lisper 9y agoIt is a bug, but it's a bug in the spec. Saying that a common mistake like dereferencing the null pointer is undefined and therefore your program can do anything is not useful behavior. The only sane design is for any attempt to dereference the null pointer to cause the program to signal an error somehow. Exactly how that happens can be left unspecified, but that it must happen cannot be unspecified in a sane design. I don't see how any reasonable person could possibly dispute this.
- jcranmer 9y ago
- kps 9y agoI have become convinced that the current screw-the-programmer interpretation of ‘undefined behaviour’ was not intended, or even imagined, by the original ANSI C committee. Within the committee's mandate to ‘standardize existing practice’, it was simply an acknowledgement that C compilers translated straightforward C code into straightforward machine code without adding safety checks, and that simple code might — in some circumstances on some machines — do very strange things. In the Rationale, the committee listed places where they intentionally diverged from existing practice. They considered “the most serious semantic change” to be requiring value-preserving (vs unsigned-preserving) integer promotion. They didn't mention ‘undefined behaviour' at all. During the standards process, Dennis Ritchie described the ‘noalias’ proposal as “a license for the compiler to undertake aggressive optimizations that are completely legal by the committee's rules, but make hash of apparently safe programs”. That's exactly what ‘undefined behavior’ has turned into. If anyone had foreseen that at the time, the reaction would have been the same: “Noalias must go. This is non-negotiable.”
- _0ffh 9y agoThis, just so much this! I've been longing for a C compiler with a "sane optimizations only"-switch like forever. I'd gladly give up on the additional couple of per cent speed improvement obsessive compulsive compiler writers managed to eke out by ignoring the source codes' obvious intentions and defending it with "but technically it's undefined behaviour"!
- staticassertion 9y agoWhat would that look like? How would this prevent me from returning a pointer to a stack frame that no longer exists, just for example? Clearly undefined behavior, but C doesn't seem capable of expressing this safely, with or without the compiler's help?
- kps 9y agoIt wouldn't make it safe; it would make it so that the consequences of returning that pointer are just the same as if you had written an assembler routine returning that pointer. On all common hardware, that would be none if the pointer is never used; on most common hardware, it would be none if the pointer is never used in a call at least equally deep — but anyway, you'd get whatever the machine does, and if you don't like it, that's your problem. In pre-UB C, the ‘spirit of C’ that C89 was intended to maintain was that you get what you wrote. By contrast, modern ‘undefined behaviour’ means that if you return that pointer — even if it is never actually used — the compiler can throw away all the code leading up to that point (including external side effects) on the grounds that it “can't happen”. You get much less than what you wrote.
- userbinator 9y agoI suspect this is another critical point: as NeverCalled may have been called from, for example, a global constructor in another file before main is run clang doesn't analyse across module boundaries --- even when it theoretically could --- so it doesn't know for certain that NeverCalled() is indeed never called. Throughout the years I've grown increasingly displeased at how compilers handle even trivial cases like this[1][2][3], and in particular their treatment of UB[4][5][6][7]; as one of the comments on the article implies, UB was unlikely intended by the standard's authors as a "you should let the most inane things happen" but more as a "do what makes the most sense". A more sensical approach to analysing this program, e.g. as employed by a human, would be to see that NeverCalled() is the only function that can write Do, but then further ascertain whether it is actually called. Since this is the entire program, it's trivial to see that it's not, and thus that possibility should also have been removed from the set of possible values for Do. Thus, Do can neither be EraseAll nor 0 --- so a "contradiction" has occurred, the code is likely bugged or the programmer intentionally wants the UB, and the sane choice at this point would be to forget about trying to optimise and just generate the obvious code. "I can't figure out how to optimise this, so I'll do the simplest thing that works." The question then becomes, why can a human see something so straightforward but the compiler can't? I think that's the deeper issue here with how the compilers like gcc/clang today work --- they're too opaque and complex, and their authors take The Holy Standard as gospel while ignoring the practical realities of their decisions. Unfortunately a lot of programmers have gotten the notion that they can rely on the compiler to do "amazing" optimisation, and therefore they can write horrible code, leading to horribly unintuitive and "overly aggressive" optimisation like this. I'm sure that me, along with quite a few others, have some ideas for how to make a C/C++ compiler which is both powerful in optimisation and code generation, but more predictable and "obvious" in terms of UB. Unfortunately, I'm also sure that we don't have the time to do it. [1] https://news.ycombinator.com/item?id=15006090 https://news.ycombinator.com/item?id=15006090 [2] https://news.ycombinator.com/item?id=9397924 https://news.ycombinator.com/item?id=9397924 [3] https://news.ycombinator.com/item?id=15188416 https://news.ycombinator.com/item?id=15188416 [4] https://news.ycombinator.com/item?id=11147598 https://news.ycombinator.com/item?id=11147598 [5] http://blog.metaobject.com/2014/04/cc-osmartass.html http://blog.metaobject.com/2014/04/cc-osmartass.html [6] https://news.ycombinator.com/item?id=7960219 https://news.ycombinator.com/item?id=7960219 [7] https://news.ycombinator.com/item?id=9809885 https://news.ycombinator.com/item?id=9809885
- valuearb 9y agoI wrote in C for 20 years and now code in Swift (which I love). Reading this reminds me that Chris Lattner created Swift because he wanted a language to do more aggressive compiler optimizations than he could do for C.
- lisper 9y agoThis is literally a religious argument. No sane person would consider this acceptable behavior if not for the fact that there is a holy text ("the standard") that says it's acceptable. Well, it's not acceptable. It is no more acceptable than, say, a car that explodes if you push the wrong button at the wrong time, which would be clearly unacceptable even if there were a document blessed by a standards committee that said otherwise. Faulty code can literally make things blow up in today's world, so there is literally (and I really do mean literally) no difference between these two scenarios. It is truly a sad reflection on the state of our profession that we are even spending time arguing about these things instead of fixing the standard so that the language it defines is actually useful for writing programs rather than just a source of material for games of intellectual one-upsmanship, to say nothing of myriad real-world problems. You'd think that decades of security breaches caused by buffer overflows would make people think, "You know, it's 2017. Maybe array dereferencing without bounds checks is a bad idea even if it does let my code run a little faster." Alas.
- a_t48 9y ago> You'd think that decades of security breaches caused by buffer overflows would make people think, "You know, it's 2017. Maybe array dereferencing without bounds checks is a bad idea even if it does let my code run a little faster." Alas. And there are dozens of languages that will let you sacrifice that bit of speed for some safety.
- deleted 9y ago[deleted]
- lisper 9y agoIf you meant to imply that this is not a problem because there are other languages one can use, I disagree. C holds a unique position in the computing world. There is an enormous corpus of C source code out there, and more is being written all the time notwithstanding that C as currently specified in not a sane language. So what C compilers do matters whether you like it or not.
- 9y ago
- IncRnd 9y agorm -rf / Using this type of code to show unintended consequences is itself a hotbed of unintended consequences! Just put in code that prints "pwned" instead of running code that would delete someone's system or home folder.
- apk-d 9y agoDo any up-to-date operating systems actually run this without the --no-preserve-root flag?
- firethief 9y agoAnyone who would compile and run random code from the internet without thinking about whether it's subtly malicious (let alone overtly destructive) has a lesson to be learned that's probably worth the data on their hard drive. If they don't have backups, that's two lessons.
- IncRnd 9y agoAgreed. But, that's not a reason for telling people to run code that would delete the entire drive.
- sbov 9y agoIf the compiler can take advantage of undefined behavior to optimize a program, what is keeping the compiler from also warning you that it is doing so?
- jjnoakes 9y agoBecause, after inlining and macro expansion and other passes, you'd get warnings about everything under the sun and they would be useless.
- kazinator 9y ago"I'm statically assuming that a pointer that can never be anything other than null actually refers to NeverCalled" is definitely worth a diagnostic.
- jjnoakes 9y agoThe compiler doesn't know the pointer isn't set, only the linker may know that. So you'd have to put a whole lot more smarts into multiple tools to get that warning. I'm not convinced it is useful.
- kazinator 9y agoThe point of the warning is precisely that the compiler is making a dangerous assumption without proof. More smarts is needed only to eliminate false positives occurrences in the warning.
- jjnoakes 9y agoIt depends on your definition of dangerous. And eliminating false positives in warnings is hugely important. Too many important warnings (more important than this one) are ignored today because people get desensitized as a result of so many false positives. Let's not add more for trivial issues like this please.
- amelius 9y agoSo how do you intentionally generate a core-dump these days?
- gizmo686 9y agoabort(); https://linux.die.net/man/3/abort https://linux.die.net/man/3/abort
- ycombobreaker 9y agoIt's probably undefined behavior, but this crashes nicely for me: int (*main)(int, char**) = 0;
- kazinator 9y agoThe problem here is that the behavior is in fact defined. It is not defined by ISO C, so it is "(ISO C) undefined behavior". But requirements for C program behavior do not only come from ISO C. It takes more requirements than just those from ISO C to make a complete implementation. On "modern", Unix-like, virtual memory platforms, we have an understanding that the null pointer corresponds to an unmapped page, and that this is the mechanism for trapping null pointer dereferences (at least ones that don't index out of that page). A compiler which interferes with this by replacing valid null pointers with garbage at point where they are dereferenced is running afoul of this platform requirement. Look, the generated code is not even pretending to use the value of the null variable. We cannot reasonably call this machine code a translation of the program's intent into machine code.
- iainmerrick 9y agoI've been playing around with Clang and I can't find any flags that avoid this problem, other than -O0. Oh, but if you add a second "NeverCalled" function that sets the pointer to something else, the optimization is skipped and you get a bus error at runtime. No flags I can find, not -Wall -Wextra -Werror, warn me at compile time that anything unusual might be going on. This is weird. I understand the explanation of the optimization, but a) isn't this rather unreasonable behavior? and b) couldn't there at least be a flag that skips this optimization, without having to disable all optimizations? Maybe this UB propagation / dead code elimination idea is so deeply ingrained in Clang's design that it would simply be impossible to selectively remove it. If so, that seems very unfortunate.