17 ms·
I've always thought that there would be more value in exactly the opposite - a pathological C compiler (or C++ compiler, POSIX operating system, etc.). The path
by tacostakohashi 11y ago
I've always thought that there would be more value in exactly the opposite - a pathological C compiler (or C++ compiler, POSIX operating system, etc.). The pathological compiler would always implement undefined behavior in the most surprising, unpredictable manner - e.g., uninitialized memory would have a properly randomized value that changes from one run to the next, overflows would result in random values, etc.
That way, any programs that are made to work on the pathological implementation are largely guaranteed to rely only on standardized behavior, and work on any other standards compliant implementation.
- 1ris 11y agoBut why? That's a way more complicated and a way less usefull.
- MereInterest 11y agoBecause if you have undefined behavior, you want your program to crash and burn as quickly as possible, so that you can remove the undefined behavior. What seems to be proposed by Dan Bernstein is not so much a "boring C compiler", but rather a superset of C which does not have any undefined or unspecified behavior.
- kazinator 11y agoSometimes if you have undefined behavior, you want the documented extension to happen. Or you want the de facto widely understood behavior to happen which has happened on every compiler you've used in 30 years: at least every compiler that was for two's complement matchines. Or every compiler on which pointers to different data types were of the same size. And so on. You also don't want to be burned by code that is relying on a common extension, when that is ported. There is a word for the compiler approach of "we're going to optimize this based on the assumption that the program is not relying on a common extension that is technically UB". That word is: malpractice. For instance, it is not unusual for C code to be targetting machines in which all objects are in the same address space, such that pointers are internally like binary numbers, allowing pointers to different objects to be compared for inequality: ptr1 < ptr2 /* does ptr1 point to a lower address than ptr2 in the one big address space? / Programmers expect this work consistently with the address structure of the machine. They don't want the expression to be somehow wrongly optimized based on the assumption that ptr1 and ptr2 must point to the same object. Implementations must honor requirements beyond those of ISO C. Such as, for instance, behaviors that those same compilers* used to define historically in their past revisions. If GCC has behaved a certain way for 25 years, and some code has come to depend on that, and then that is suddenly taken away, such that entire GNU/Linux distros continue to compile, but break in random places because of that detail, the fault for the regression lies 105% with the doofus who made the GCC commit. Even if that behavior is not defined by ISO C.
- quotemstr 11y ago> which pointers to different data types were of the same size Member function pointer size varies widely between ABIs.
- kazinator 11y agoI suspect you're talking about "pointer to member", which doesn't exist in C; it is a C++ concept. This is not a pointer, but a kind of indirection mechanism dressed up in pointer syntax. I've never seen C++ code which depended on the internals of pointers-to-member, or assumed they had the same size as other kinds of pointers. A pointer to member is actually a complicated offset into an offset into some structure associated with an object's class. This offset can be "dereferenced" with respect to objects of different types in the same hierarchy, taking into account multiple inheritance and virtual bases, etc. A pointer to a Base::Foo function, can be applied against a Derived object, such that it resolves to the correct code, even though Base is the third base of Derived, and so Foo is in a totally different vtable position in Derived compared to Foo. (The pointer cannot be a blind integer offset.) In C, we would make a struct containing function pointers, and a simple "pointer to member" would just be an integer "offsetof(our_struct_type, some_function)". Then worry about it later if someone wanted multiple inheritance. :)
- quotemstr 11y agoNevertheless, it is a kind of pointer (it's called a "pointer") with a non-uniform size. That its internal structure is composite (just like a segmented 80286 far pointer) is irrelevant.
- deleted 11y ago[deleted]
- gherkin0 11y agoI think the idea is that any reliance on undefined behavior would blow up in your face and immediately cause a bugs, rather than working correctly with some compilers and not with others.
- kazinator 11y agoThe problem is that undefined behaviors are used for valid extensions. "Undefined behavior" doesn't mean "error". For instance including a platform header like #include <fortran.h> is undefined behavior. ISO C has some requirements there: if the header is found, then the directive is replaced with the preprocessing tokens from reading that header (and so if there are syntax errors there, they have to be diagnosed, and so on). But ISO C specifies no requirements as to what <fortran.h> contains, or whether it exists. On one implementation, it might cause the rest of the translation unit to be treated as a Fortran program. On an other implementation, it might bring in some declarations related to a Fortran interoperability. On yet another, translation might stop with "header not found: fortran.h". UB is simply "behavior upon use of a nonportable or erroneous construct, of erroneous data, or of indeterminately-valued objects for which [ISO C] imposes no requirements". Nonportable is not necessarily erroneous!
- pmjordan 11y agoThere is a difference, in the standard, between undefined and implementation-defined (unspecified) behaviour though.
- kazinator 11y agoThere is also a difference between "implementation-defined" and "unspecified". Look it up! The former is similar to the latter, only it must be documented. Unspecified behaviors imply some choice from among alternatives which need not be documented. (Like the order in which the argument expressions of a function are evaluated.)
- tacostakohashi 11y agoNot really. There are a lot of platforms that already have a compiler that the vendor claims is standard, so being able to ensure programs rely only on the standard would make it easy to target all of these existing platforms and compiler implementations. The "boring" compiler doesn't exist yet, and would need to be implemented on every platform that currently has a standard compiler to be as useful.
- dxhdr 11y agoI'm not sure this would be helpful, working with any unpredictable system is a nightmare. The compiler emitting warnings of undefined behavior being utilized would be the best solution, though it would not be able to identify all scenarios. Input fuzzing would likely get you the rest of the way there.
- octo_t 11y agothe problem is that the compiler assumes undefined behaviour can never happen ALL the time. Signed integer overflow is the most common one, since it means you can hoist 32bit ints into 64bit ints so they fit in one register or similar and save hitting the stack.
- deleted 11y ago[deleted]
- derefr 11y agoI'd personally be fine with being forced to specify what sort of undefined behaviors I want to allow to be "assumed I didn't mean"; it's not a problem for me if some code refuses to compile until I e.g. type-annotate my u32s with "they're not going to overflow, don't worry." I was about to say that "this would break a ton of other projects"—but actually, it seems like it'd be fine as long as it was an opt-in -W switch.
- __s 11y agoIt could detect all scenarios by compiling to a C virtual machine which checks for UB at runtime
- tacostakohashi 11y agoEmitting warnings and errors mostly works fine for undefined behaviors, the real problem is with implementation-defined behaviors. Let's say that, on the pathological compiler, it's documented that sizeof(int) is 4, and chars are signed if the number of seconds in the current minute is even, otherwise sizeof(int) is 5 and chars are unsigned. If your code compiles and works with either setup, it's probably correct and portable. It's not really feasible for programmers to avoid implementation-defined things like sizeof(int) varying across implementations, not is it feasible for compilers to detect assumptions made about the implementation's definition in the program. The existence of a pathological compiler would make it far easier to write programs that work across all implementations. It's true that working with unpredictable systems is a nightmare, but that's precisely the nature of undefined and implementation-defined behavior in C. The existence of a pathological compiler would merely allow developers to work through these issues on a single machine, which would be an improvement on needing to obtain a variety of compilers and platforms test with to find these issues.
- GFK_of_xmaspast 11y agoIt's hard enough getting people to turn warnings on.
- tacostakohashi 11y agoThis is true - in general, a warning means the code is non-standard or ambiguous, so the compiler is falling back to some default or non-standard interpretation. Then, people come to rely on that, and, like you say, not turn warnings on. So, I say that's not a useful class of behavior, and in some cases (trying to write secure / portable software) it would be better if that didn't exist.
- ZoF 11y agoThis makes zero sense to me. Where would this be beneficial? Why not just fuzz inputs if this is your goal?
- tacostakohashi 11y agoThat's what this is, keeping in mind that the compiler's implementation of undefined behavior is an input that should be fuzzed. Think of, say, integer overflow. The problem is that typical compilers have a consistent (but non-standard) implementation, so it can't be fuzzed without some compiler help.
- deleted 11y ago[deleted]
- ufo 11y agoTests can show the presence of bugs but not their absence. The idea here is to create a compiler for C (or perhaps a C dialect) so that nasty undefined behavior is guaranteed to not occur.
- __jal 11y agoWe actually (almost) have that. It is called gcc. A long time ago, when encountering "undefined" behavior, it would try to launch Nethack[1]. (It is valid behaviour, according to the spec.) Seriously, ask around. There was a multi-month fat-chewing about gcc's pathological interpretation of "undefined" earlier this year on the Cryptography list, and tends to come up anywhere C programmers with an interest in security drink. [1] http://feross.org/gcc-ownage/?1?1 http://feross.org/gcc-ownage/?1?1
- _kst_ 11y agoThat's inaccurate, according to the page you linked to (and to my memory of the incident). gcc 1.17 would invoke nethack (or one of several other similar games, if available) if it saw an unrecogized #pragma directive. According to the C standard, the behavior of a #pragma not followed by STDC is implementation-defined, not undefined -- which means that an implementation is required to document its behavior. (I presume that gcc did so.) gcc did not play launch Nethack in response to undefined behavior in general, or in response to anything other than an unrecognized #pragma directive.
- qb45 11y agoI don't know if it's guaranteed to bark on all possible cases of UB, but there we go: $ cat test.c int main() { return *(int*)0; } $ gcc test.c -fsanitize=undefined -o test $ ./test test.c:2:9: runtime error: load of null pointer of type 'int' Segmentation fault I'm pretty sure I've seen similar tool based on clang as well. And there is valgrind.
- an_d_rew 11y agoJust for the hardware-minded: dereferencing null is perfectly fine on Cortex-M processors and is well-defined. Yes, that's hardware, but a nice place where hardware ain't C. I'm just sayin'...
- qb45 11y agoDo you mean some quirk unique to these CPUs or do they simply have RAM mapped at this address? The latter is nothing unusual, x86 has it as well.
- CountSessine 11y agoLike many embedded cores, Cortex-M's don't have an MMU - everyone sees physical memory. In fact, I believe that's the chief distinguishing feature of the 3 core levels - Cortex-M's have no MMU, Cortex-A's have memory paging, and I think Cortex-R's are in between and have memory segments like MIPS. So if the board designer wired everything up to map RAM at address 0, then yes, you'd be able to dereference a pointer to address 0. Completely beside the point, though. Dereferencing NULL is always undefined behaviour in C.
- amluto 11y agoI'm a bit curious how a pointer with value 0 ended up being banned by C. Specifically, on x86 real mode (which includes modern systems early in boot before switching to protected mode), there's a hardware defined data structure at address zero: the "Interrupt Vector Table". Maybe everyone who wrote real-mode systems software either ignored that bit of the C standard, or they fudged around it (IIRC the IVT is an array and the first entry doesn't matter, so you can start a few bytes past zero), or they used assembly to access it.
- ChuckMcM 11y agoBut that is the thing, undefined behavior just says that the compiler takes no position on what should happen. So a boring compiler that refuses to compile code which would have undefined behavior is sufficient. It makes idomatic C a bit more challenging but if you've read through Dan's code you know that he's all about picking a style and sticking with it. And for that set of constraints, its all defined (and boring because there is very little, if any, optimization going on insider the compiler) So sub-optimal but predictable and safe code given that you change the preference bit to safety over speed.
- deleted 11y ago[deleted]
- ProblemFactory 11y agoA lot of undefined behaviour cannot be identified at compile time. For example, adding two signed integers is undefined if it overflows, but perfectly well defined if it does not. A safe compiler will have to add checks to guard against undefined behaviour happening at runtime.
- Pxtl 11y agoUm, sounds like the most appropriate response would be a low-performance C compiler/interpreter that enforces crashes on all run-time undefined behaviors, and refuses to compile for any compile-time checkable ones.
- kenko 11y agoWhy is that more appropriate than just defining all the currently undefined/implementation-specific behavior?
- Pxtl 11y agoBecause enforcing undefined behavior on this test-compiler means you can build your software on any standards-compliant compiler and have it work the same. In theory.
- CountSessine 11y agoIMO both approaches would be useful. One nice thing about outlawing undefined behaviour rather than defining it, though, it that you'll be writing portable code - if you have a code base that's filled with all all sorts of once-was-undefined-but-now-is-defined behaviour, you can never build that code on another compiler.
- deleted 11y ago[deleted]
- nwmcsween 11y agoUbsan already does this, there is a flag that controls abort on UB.
- pekk 11y agoUnfortunately, nobody will use that compiler. They will continue to use the high-performance compilers, because this is what happened decades ago.
- haberman 11y agoIf the runtime can detect undefined behavior, why do something random? Just report the undefined behavior to the user. Now you have UBSan: https://www.chromium.org/developers/testing/undefinedbehaviorsanitizer https://www.chromium.org/developers/testing/undefinedbehavio...
- nickpsecurity 11y agoFunny idea but best to leave that to code analysis tools. So, your compiler works as expected and works well. That's why you use it. Other peoples' work like shit due to bugs and undefined behavior. So, a coding style plus analysis tools to reinforce it make your code work well with others' compilers while you still use The Right Thing. ;)
- ape4 11y agoIt could just crash with an error.
- lazyjones 11y ago> a pathological C compiler I had the same opinion at first and would agree if UB was mostly possible to avoid in meaningful programs, but it isn't. The main casualty would be the reason why this is being discussed at all: compilers need to make very specific assumptions on UB to enable some optimizations and if UB goes away, so do the optimizations. So we get the performance of the "boring" compiler anyway, just with more effort on behalf of the programmer. DJB is (as always, I guess) right, C simply isn't really suited to optimizations based on UB. What'd be the big deal if compiler developers just stopped inflicting these on C programmers and focused on optimizing compilers for better-defined languages (with range types, specified overflow behavior etc.) instead?
- duaneb 11y ago> What'd be the big deal if compiler developers just stopped inflicting these on C programmers and focused on optimizing compilers for better-defined languages (with range types, specified overflow behavior etc.) instead? This is non-trivial. These definitions come with tradeoffs that fundamentally alter the language. However, this is basically what rust is.
- xamuel 11y ago>inflicting these on C programmers You're acting like it's unambiguously bad, but that's not true. For example, I might create a macro DEREF_AND_FREE(x) which expands to if(x)free(*x). Often times, I'll lazily use that in places where I know x isn't null. It's more readable and maintainble than splitting the macro up into two separate macros, DEREF_AND_FREE_IF_NONNULL and DEREF_AND_FREE_I_KNOW_ITS_NONNULL. These UB-based optimizations which are "inflicted" on me mean that I don't take the performance hit for my laziness.
- duaneb 11y agoHow is this any better than simply noting undefined behavior as an error?
- 010a 11y agoGo does this in at least one way. The iteration order of maps is undefined in the language spec. The go runtime automatically and purposefully randomizes the order of items in a hash so programmers can't rely on the arbitrary ordering it may have at any time. https://blog.golang.org/go-maps-in-action https://blog.golang.org/go-maps-in-action
- WalterBright 11y agoI used to use with C a replacement for malloc() that would initialize the memory to 0xDEADBEEF. It was amazing how many bugs were forced into the open. These days, valgrind is a much better option for doing this.