4 ms·

> I think the amount of undefined behavior in C--and that undefined behavior is necessary to compensate for its design mistakes (e.g. signed overflow)--disquali

> I think the amount of undefined behavior in C--and that undefined behavior is necessary to compensate for its design mistakes (e.g. signed overflow)--disqualifies it from being a "great language". The undefined behavior is there primarily to account for the variation in platforms. Higher-level languages avoid this by hiding it in a platform-specific compiler or VM, which in turn limits the number of platforms the code can actually be run on. > Empirically, virtually every single large C program in existence depends on undefined behavior to continue working. Citation needed. I get that undefined behavior is hard to spot, but having spent years writing cross-platform C I cannot accept that every large program depends on it without some evidence.

catnaroek 10y ago
> The undefined behavior is there primarily to account for the variation in platforms. Honest question: Doesn't implementation-defined behavior suffice to address variations across platforms? Undefined behavior has far more drastic consequences: compilers are allowed to assume it simply won't happen.
- vonmoltke 10y ago
  On further review, I think I was conflating undefined behavior with implementation-defined behavior. As pcwalton pointed out, though, in the early days there wasn't really a distinction between the two.
pcwalton 10y ago
> The undefined behavior is there primarily to account for the variation in platforms. Higher-level languages avoid this by hiding it in a platform-specific compiler or VM, which in turn limits the number of platforms the code can actually be run on. It may have started this way in the committee (though that isn't true for e.g. type based alias rules), but nowadays it's depended on for performance. You lose a lot of ability to reason about loop trip count if signed overflow is defined, for example. > Citation needed. I get that undefined behavior is hard to spot, but having spent years writing cross-platform C I cannot accept that every large program depends on it without some evidence. The probability that a program depends on any of (nonexhaustive list) (a) inspecting the value of a dangling pointer; (b) taking the offset from NULL; (c) type-punning through a union; (d) type-punning through any type other than char; (e) signed overflow; (f) order of operations between sequence points quickly approaches 1. Here's a very interesting survey investigating this: https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-discussion.html https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-disc...
- vonmoltke 10y ago
  > It may have started this way in the committee (though that isn't true for e.g. type based alias rules), but nowadays it's depended on for performance. You lose a lot of ability to reason about loop trip count if signed overflow is defined, for example. I agree it is depended on in many cases, and should not be. That is on the programmers though, not the language designers. The behavior does not exist to cover deficiencies in the language but those of its users. > The probability that a program depends on any of (nonexhaustive list) (a) inspecting the value of a dangling pointer; (b) taking the offset from NULL; (c) type-punning through a union; (d) type-punning through any type other than char; (e) signed overflow; (f) order of operations between sequence points quickly approaches 1. Just on your short list I can tell you the probably of any of our C code depending on that approached zero: > inspecting the value of a dangling pointer Our code was malloc once, free never. Dangling pointers were by definition impossible. > taking the offset from NULL > type-punning through a union > type-punning through any type other than char > signed overflow > order of operations between sequence points Our coding standards covered all of those, so their existence would be a code review failure. We weren't specifically writing to MISRA C, but I will wager that your statement is not accurate for any project that follows it or any similar standard. Yes, it is a lot of work that you don't need to do with a more controlled and less flexible language. I'm not going to challenge that. I'm just challenging the idea that nobody is writing C that avoids undefined behavior.
  - catnaroek 10y ago
    > Our code was malloc once, free never. Dangling pointers were by definition impossible. Dangling pointers can arise in other ways: use an uninitialized pointer variable, take the wrong offset from an otherwise valid pointer, etc.
    - vonmoltke 10y ago
      Those are indeterminate values but they aren't dangling pointers. Dangling pointers are specifically references to an object that has been freed. Plus, the cases you highlighted are programming errors, not reliance on undefined behavior.
stcredzero 10y ago
Higher-level languages avoid this by hiding it in a platform-specific compiler or VM, which in turn limits the number of platforms the code can actually be run on. False. Squeak Smalltalk runs with a bit-identical model on a ridiculous number of platforms. (which is to say OS/ISA combinations) Though to be fair, it accomplished this trick by having the VM written in a Smalltalk subset that gets translated into C, which is then compiled by local compilers.
nkurz 10y ago
Citation needed. I get that undefined behavior is hard to spot, but having spent years writing cross-platform C I cannot accept that every large program depends on it without some evidence. I'm not finding an academic citation, but perhaps a respected academic's blog counts? I think John Regehr is the likely source of these claims. In his words: My guess is that most or all executions of GCC and LLVM (and basically all other large programs) would be undefined. http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761 He starts the post by stating that the "majority of the programs in SPEC CINT 2006 execute undefined behavior during the course of a 'reportable' run". A majority is definitely not "all", but the SPEC programs are also not "large": Let’s consider the SPEC benchmark programs. Making them fast is something that most compiler vendors would like to do. Using IOC we found that a majority of the programs in SPEC CINT 2006 execute undefined behavior during the course of a “reportable” run. For large programs, he then looks at the GCC and Clang compilers themselves, and concludes: If we consider them to be written in C99, LLVM/Clang 3.1 and GCC (SVN head from July 14 2012), running on Linux on x86-64, do not appear to admit any non-trivial, non-erroneous executions. I expect the same is true of most or all previous versions of these compilers. IOC tells us that these compilers execute undefined behaviors even when compiling an empty C or C++ program with optimizations turned off. Lest you think that the undefined behavior is caused by inappropriately forcing C99 on earlier code, he goes on to explain that the problems probably exist there too, but that the lack of tooling for earlier dialects prevents proper analysis: My guess is that most or all executions of GCC and LLVM (and basically all other large programs) would be undefined by C89 (and not just C99) but we can’t tell for sure without using a comprehensive undefined behavior checker for C89. This tool does not exist. I confess to not personally having the gumption necessary for cramming GCC or LLVM through the best available dynamic undefined behavior checkers: KCC and Frama-C. His overall message (which I think is true and merits significant reflection) is that C's utility as a real world language is based on unspoken agreements with compiler writers to use some degree of judgment and restraint when making optimizations based on undefined behavior. Rather than a standardized language defined by the written specifications, we are dependent on an "undocumented dialect of C where undefined behavior can sometimes be exploited and sometimes not": This means that a correct C/C++ compiler could generate code that is almost infinitely faster than the code generated by current compilers. The SPEC harness would of course complain that the compiled programs produce the wrong results, but this is actually a bug in the SPEC harness, not a bug in the compilers. If we insist that a compiler turns these SPEC programs into executables that produce the specified results, then we are effectively mandating that this compiler implements a bizarre, undocumented dialect of C where undefined behavior can sometimes be exploited and sometimes not. http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761

catnaroek 10y ago

> The undefined behavior is there primarily to account for the variation in platforms. Honest question: Doesn't implementation-defined behavior suffice to address variations across platforms? Undefined behavior has far more drastic consequences: compilers are allowed to assume it simply won't happen.

vonmoltke 10y ago
On further review, I think I was conflating undefined behavior with implementation-defined behavior. As pcwalton pointed out, though, in the early days there wasn't really a distinction between the two.

pcwalton 10y ago

> The undefined behavior is there primarily to account for the variation in platforms. Higher-level languages avoid this by hiding it in a platform-specific compiler or VM, which in turn limits the number of platforms the code can actually be run on. It may have started this way in the committee (though that isn't true for e.g. type based alias rules), but nowadays it's depended on for performance. You lose a lot of ability to reason about loop trip count if signed overflow is defined, for example. > Citation needed. I get that undefined behavior is hard to spot, but having spent years writing cross-platform C I cannot accept that every large program depends on it without some evidence. The probability that a program depends on any of (nonexhaustive list) (a) inspecting the value of a dangling pointer; (b) taking the offset from NULL; (c) type-punning through a union; (d) type-punning through any type other than char; (e) signed overflow; (f) order of operations between sequence points quickly approaches 1. Here's a very interesting survey investigating this: https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-discussion.html https://www.cl.cam.ac.uk/~pes20/cerberus/notes50-survey-disc...

vonmoltke 10y ago
> It may have started this way in the committee (though that isn't true for e.g. type based alias rules), but nowadays it's depended on for performance. You lose a lot of ability to reason about loop trip count if signed overflow is defined, for example. I agree it is depended on in many cases, and should not be. That is on the programmers though, not the language designers. The behavior does not exist to cover deficiencies in the language but those of its users. > The probability that a program depends on any of (nonexhaustive list) (a) inspecting the value of a dangling pointer; (b) taking the offset from NULL; (c) type-punning through a union; (d) type-punning through any type other than char; (e) signed overflow; (f) order of operations between sequence points quickly approaches 1. Just on your short list I can tell you the probably of any of our C code depending on that approached zero: > inspecting the value of a dangling pointer Our code was malloc once, free never. Dangling pointers were by definition impossible. > taking the offset from NULL > type-punning through a union > type-punning through any type other than char > signed overflow > order of operations between sequence points Our coding standards covered all of those, so their existence would be a code review failure. We weren't specifically writing to MISRA C, but I will wager that your statement is not accurate for any project that follows it or any similar standard. Yes, it is a lot of work that you don't need to do with a more controlled and less flexible language. I'm not going to challenge that. I'm just challenging the idea that nobody is writing C that avoids undefined behavior.
- catnaroek 10y ago
  > Our code was malloc once, free never. Dangling pointers were by definition impossible. Dangling pointers can arise in other ways: use an uninitialized pointer variable, take the wrong offset from an otherwise valid pointer, etc.
  - vonmoltke 10y ago
    Those are indeterminate values but they aren't dangling pointers. Dangling pointers are specifically references to an object that has been freed. Plus, the cases you highlighted are programming errors, not reliance on undefined behavior.

stcredzero 10y ago

Higher-level languages avoid this by hiding it in a platform-specific compiler or VM, which in turn limits the number of platforms the code can actually be run on. False. Squeak Smalltalk runs with a bit-identical model on a ridiculous number of platforms. (which is to say OS/ISA combinations) Though to be fair, it accomplished this trick by having the VM written in a Smalltalk subset that gets translated into C, which is then compiled by local compilers.

nkurz 10y ago

Citation needed. I get that undefined behavior is hard to spot, but having spent years writing cross-platform C I cannot accept that every large program depends on it without some evidence. I'm not finding an academic citation, but perhaps a respected academic's blog counts? I think John Regehr is the likely source of these claims. In his words: My guess is that most or all executions of GCC and LLVM (and basically all other large programs) would be undefined. http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761 He starts the post by stating that the "majority of the programs in SPEC CINT 2006 execute undefined behavior during the course of a 'reportable' run". A majority is definitely not "all", but the SPEC programs are also not "large": Let’s consider the SPEC benchmark programs. Making them fast is something that most compiler vendors would like to do. Using IOC we found that a majority of the programs in SPEC CINT 2006 execute undefined behavior during the course of a “reportable” run. For large programs, he then looks at the GCC and Clang compilers themselves, and concludes: If we consider them to be written in C99, LLVM/Clang 3.1 and GCC (SVN head from July 14 2012), running on Linux on x86-64, do not appear to admit any non-trivial, non-erroneous executions. I expect the same is true of most or all previous versions of these compilers. IOC tells us that these compilers execute undefined behaviors even when compiling an empty C or C++ program with optimizations turned off. Lest you think that the undefined behavior is caused by inappropriately forcing C99 on earlier code, he goes on to explain that the problems probably exist there too, but that the lack of tooling for earlier dialects prevents proper analysis: My guess is that most or all executions of GCC and LLVM (and basically all other large programs) would be undefined by C89 (and not just C99) but we can’t tell for sure without using a comprehensive undefined behavior checker for C89. This tool does not exist. I confess to not personally having the gumption necessary for cramming GCC or LLVM through the best available dynamic undefined behavior checkers: KCC and Frama-C. His overall message (which I think is true and merits significant reflection) is that C's utility as a real world language is based on unspoken agreements with compiler writers to use some degree of judgment and restraint when making optimizations based on undefined behavior. Rather than a standardized language defined by the written specifications, we are dependent on an "undocumented dialect of C where undefined behavior can sometimes be exploited and sometimes not": This means that a correct C/C++ compiler could generate code that is almost infinitely faster than the code generated by current compilers. The SPEC harness would of course complain that the compiled programs produce the wrong results, but this is actually a bug in the SPEC harness, not a bug in the compilers. If we insist that a compiler turns these SPEC programs into executables that produce the specified results, then we are effectively mandating that this compiler implements a bizarre, undocumented dialect of C where undefined behavior can sometimes be exploited and sometimes not. http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761