12 ms·

“Fiercely resist any further broadening of the scope of the C UB problem”

118 points by Sindisil 11y ago

forrestthewoods 11y ago
"But sadly it seems that the notion that our most basic and widely-used programming language should be one that's fit for programming in is not yet fully accepted." Ouch. I usually hear that about C++, not C. =[
- twoodfin 11y ago
  My guess is that many of the optimizations causing increasing trouble were introduced to improve C++ code performance in pursuit of the "zero-cost abstraction" goal. But many are just as applicable to C (and may even be below the level that a compiler like gcc or clang knows the difference).
  - ArkyBeagle 11y ago
    The whole "zero-cost abstraction" thing puzzles me. Abstraction is a means to an end, not an end unto itself.
    - rbancroft 11y ago
      We want abstractions to make things easier to do, but we don't want to pay for them with CPU cycles... that's what a zero-cost abstraction means. It's not about adding in useless abstractions. The ideal is software that is faster to write and easier to understand and maintain, while offering uncompromised execution performance.
haberman 11y ago
I can see both sides of this. But looking at this argument: > There are two ways to evaluate the the C specification's rightness and properness. [...] The second is to ask what is most useful. And there again the C committee have clearly failed. Just two days on HN we saw this article: https://news.ycombinator.com/item?id=11468603 https://news.ycombinator.com/item?id=11468603 In which it says: > no matter the kind of software, performance is almost always worse than our customers would like it to be. That is why all of this is happening. There is a market demand for performance. Compilers that increasingly exploit UB in C is just a manifestation of that market demand.
- michaelt 11y ago
  There is a market demand for performance. You could fulfil that demand without breaking existing code by making the riskier optimisations configurable and turned off by default.
  - webkike 11y ago
    They are usually. Most people don't change the -03 in their build files after a compiler update though.
    - ktRolster 11y ago
      -O4 exists for 'risky' optimizations. -O3 is for optimizations that are mostly not risky. -O2 is for optimizations that don't involve a speed/size tradeoff -O1 is for optimizations that don't slow your compiler down too much Obviously these are fuzzy categories, but -O4 is where "undefined behavior" optimizations belong. Alternately, each individual optimization has its own flag, and can be turned on individually.
      - haberman 11y ago
        I've never heard of -O4 for "risky" optimizations, and I can't find any reference to this. Do you have one?
        ktRolster 11y ago
        If you search for "gcc O4" you'll find some references (it's tricky because if you search for -O4 Google will search for everything without O4). Currently, there are no optimizations that fit in that category, but as recently as 2003 I remember seeing it in the man pages. I believe there was -O5 and -O6 too, but I don't trust my memory enough to be certain.
        haberman 11y ago
        The results for that query all seem to say that "gcc -O4" doesn't exist. There is one proposal from Chris Lattner for adding this to Clang, but it's not clear that this ever happened.
deleted 11y ago
[deleted]
Glyptodon 11y ago
UB being the various "undefined behavior" bits of C?
- cpeterso 11y ago
  Yes. Compiler developers like it; everyone hates it. :)
  - twoodfin 11y ago
    You like it when aliasing rules allow your compiler to copy the target of a pointer into a register, rather than read it from memory over and over in case an intervening write through another pointer could have overwritten it.
hinkley 11y ago
I have to say, strictly as an uninformed outsider, this bit of nastiness is just pushing Go, Rust and even Swift higher on my priorities list. I don't think I'm the only one.
- ktRolster 11y ago
  Well then, don't look too deeply at the nastiness in Go, Rust, and Swift.
  - dsfuoi 11y ago
    Can you elaborate? Hopefully they don't have straight UB do they?
    - ktRolster 11y ago
      Hopefully they don't have straight UB do they? Yes, most languages do, unless they are formally defined (ML is formally defined, but most other languages are not. In this book http://www.amazon.com/Masterminds-Programming-Conversations-Creators-Languages/dp/0596515170 http://www.amazon.com/Masterminds-Programming-Conversations-... several of the language creators say that fully defining the language formally is not worth the effort).
    - zokier 11y ago
      Well, Rust for example does not have a formal well-defined behavior to start with, so its kinda fuzzy about UB too. And of course `unsafe` is another story altogether.
      - mastax 11y ago
        Can you elaborate? The rust docs [1] seem to disagree: Unlike C, Undefined Behavior is pretty limited in scope in Rust. All the core language cares about is preventing the following things: - Dereferencing null or dangling pointers - Reading uninitialized memory - Breaking the pointer aliasing rules - Producing invalid primitive values: - dangling/null references - a bool that isn't 0 or 1 - an undefined enum discriminant - a char outside the ranges [0x0, 0xD7FF] and [0xE000, 0x10FFFF] - A non-utf8 str - Unwinding into another language - Causing a data race And all of those things require `unsafe`, so safe rust cannot do any of them (barring compiler or unsafe library or OS bugs). Edit: And I must admit I don't know much about language theory or formal definition, but there is also a self-described formal grammar [2]. [1]: https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html [2]: https://doc.rust-lang.org/grammar.html https://doc.rust-lang.org/grammar.html
jabl 11y ago
The obvious problem with the "The second is to ask what is most useful" approach is that if you ask 1000 C programmers, 990 are not clueful enough to give any sane answer and will pollute your answer database with noise, and of the remaining 10, 5 are language lawyers and favor approach #1 anyway, and the other 5 will give 20 different answers (Note: above numbers completely made up). So then the compiler developers give up and implement stuff 1) according to the standard document 2) such that performance on SPEC cpu is maximized.
- _delirium 11y ago
  A related comment: http://blog.regehr.org/archives/1287 http://blog.regehr.org/archives/1287
  - jabl 11y ago
    Indeed. The only realistic way out of this conundrum AFAICS is some kind of consensus around "friendly C"/"boring C" or such that compilers then can implement, whether it's an annex to the official C standard or some other kind of spec. That, or then everybody switches to Rust. :) (hey, I can daydream can I?).
    - pcwalton 11y ago
      And everyone will have to accept the performance loss--and there will be performance loss. This is acceptable for many applications, don't get me wrong. But it will be a tough sell. If you want to get people using a Friendly C, you need to start convincing the biggest users of C and C++ that they should give up performance for a simpler dialect of the language. Like politicians responding to voters, compiler authors respond to what their customers demand. Up to now, their customers have demanded performance. It's not their fault for listening to them.
      - dsfuoi 11y ago
        Performance of real code, not micro-benchmarks, won't change. And this can work both ways. Friendly C would almost certainly forbid non-compatible type pointer castings, which would actually enable additional optimizations.
ktRolster 11y ago
This narrative of `fault' has two very serious problems. That can be said almost any time people are trying to find fault. Better to just look for a solution, and not worry about whose fault it is.
cpeterso 11y ago
What are the particular compiler behaviors or optimizations they would disable?
- twoodfin 11y ago
  A quick look at the first example provided suggests the restriction on comparing pointers to internal members of distinct structures.
- SQLite 11y ago
  How about allowing pointers to be compared against NULL after they have been passed to free(): "free(x); if( x!=NULL ){...}" That was completely harmless (and a useful idiom) for 45 years, and now suddenly it is flagged as UB and has to be changed. Why? Wouldn't it be great if "memset(0,p,0)" was a harmless no-op? It was for time out of mind. But no more. For bonus points: Can we have a #pragma that tells the compiler to abort with an error if the target machine uses any representation for signed integers other than twos-complement?
  - dsfuoi 11y ago
    Why? Because programmers learned that it is correct, even though it wasn't.(Or at least it stopped being, in C99 an on.) Here is a car analogy. It is like learning to drive and thinking that you are allowed to drive over a yellow-turning-to-red light. However the rules say, you should stop if you are physically able to. In reality almost everyone tries to get over than yellow. On a rare occasion they get spotted and pay the fine. C strives for maximum performance, it will not check things for you if you don't ask it to. Having a library function that performs those checks for everyone, would go against that rule. Why would someone else have to pay the performance penalty for you? Write a wrapper or a macro that performs the check, couple of lines, it is that easy. If you strive to write portable C code it should work regardless of signed representation. C defines all the range macros for types, and gives you types that guarantee certain ranges. Assuming you use those types and macros, I'm really curious what incantations, that couldn't been solved in a portable manner, require you to know the signed representation.
  - 11y ago
haberman 11y ago
One area that worries me that I never see anyone talk about is LTO between C and C++. How do you even reason about that? I asked a C++ and compiler expert about this once and he told me: "I believe both GCC's and LLVM's LTO will happily cross this barrier, so it doesn't offer you any real protection from their optimizers." There is no single standard that defines what UB is for a mixture of C and C++. There are clearly some parts of C++ that are trying to improve interoperability with C, like "standard layout" classes. The best you can do is try to simultaneously follow the rules for both languages when you mix the two.
- Kristine1975 11y ago
  Why not flag each function with "C" or "C++" depending on where they originate from? The linker can then perform only optimizations that are valid for the respective language.
- DSMan195276 11y ago
  Personally, I think this is blown out of proportion a bit - There are things to complain about, but LTO problems are really only caused by other issues. But with that said, what you described isn't really a worry: LTO is done on the compilers internal representation of the code (IE. GIMPLE or LLVM IR). This representation is generated based on the rules of each language, and optimizations are performed on this representation instead of the original source representation. Both C and C++ (and anything else) are converted into these representations. LTO simply keeps the GIMPLE or LLVM IR around until link time, and then when the program is linked optimizations are performed over the entire representation of the program. Crossing the language barrier shouldn't be a problem, because the GIMPLE has it's own rules to follow to make sure everything still functions the same. Once you reach this point both languages are already compiled in the practical sense, they're just not actual machine code yet. I would expect however, that because of the differences there are far less optimization opportunities to be taken advantage of.
  - haberman 11y ago
    Ok let's look at a specific example then. // foo.c #include <stdlib.h> typedef struct { int x; } s; s *make_s() { s *ret = malloc(sizeof(*ret)); ret->x = 5; return ret; } void free_s(s *val) { free(val); } // bar.cc // This class is standard-layout and matches "s" from C. class C { int getX() { return x_; } private: int x_; } extern C* make_s(); extern void free_s(C* c); int main(void) { C* c = make_s(); int ret = c->getX(); free_s(c); return ret; } C++ says you can only call a method on an object whose lifetime has begun. Can C begin the lifetime of a C++ object?
dsfuoi 11y ago
C is designed to be capable of running of vastly different architectures, allowing the programmer to use that flexibility when it actually isn't needed, or is harmful. I think the C coding mindset should be this: if an approach requires code that isn't unambiguously defined, change the approach. If this means more boring coding, to circumvent cute tricks, so be it. If you need that extra 5% speed, use assembly instead of bending C.
- astrobe_ 11y ago
  The problem is that people often don't even know what is UB and what is not. People don't even always know what is portable and what isn't, to begin with. Some even seem to believe that "it's written in C, so it's portable".
  - dsfuoi 11y ago
    I strongly agree. There is a serious lack of respect of the C rules even among those who teach and this is passed onto their students. It is really hard to convince them to change ways after that. A couple months ago, as a curiosity, I watched a few videos in a series on programming in C in Windows environment. The teacher was a serious programmer, but the first thing that went out the window was strict aliasing. After that assumptions of integer sizes and range started creeping in. It came apparent that the teacher knew C, but only superficially, signedness, integer promotions and usual arithmetic conversions were treated like a nuisance. If the code compiled and ran, it was good. Those videos were the first C programming experience for at least several hundred people.
sneak 11y ago
Amazing that in 2016, the best programmers in the world have not chosen to solve the "display mailing list archives on a webpage without terrible line wrapping" problem yet.
- oldmanjay 11y ago
  It was hard to think about, it should be hard to read
robertelder 11y ago
The issue of creeping optimizations causing problems far in the future is one of the reasons that I suggested using unsigned intergers in C as much as possible: "Compiler authors will likely support this as long as possible, but the peer pressure of needing things to go faster and faster will likely push them to exploit more and more undefined behaviour to their advantage in the future. Their argument will be: 'After all, who has sympathy for those who don't follow the standard?'." From: http://blog.robertelder.org/signed-or-unsigned-part-2/ http://blog.robertelder.org/signed-or-unsigned-part-2/ Lots of people disagree with me, but its nice to have less UB issues to worry about far in the future.
ComputerGuru 11y ago
I don't know if it's because I convinced myself a long time ago that I'll never be smart enough to write correct C/C++ code (though that hasn't (yet) stopped it from being my primary development language) or if it's because I just because I'm a secret masochist, but I disagree with this proposal/suggestion. Regardless of who is right and who is wrong in this matter, I think if everyone took a step back we could at least agree that it makes absolutely no sense to fix this on a (single Linux) distribution level. For Debian to configure/patch compilers on their platform to "narrow" undefined behavior is insane and ineffectual. Software isn't "validated" on/against a particular OS, it's validated on a compiler basis. Breaking this assumption introduces a massive schism. While Debian is an amazing distro with plenty of clout (I'm a FreeBSD guy, but Debian comes second), it's terrifying to imagine a new generation of "cross-platform" C/C++ software that can only be verified working on Debian (or with Debian's fork/re-configured compiler). We've come so close to making truly cross-platfrom C++ code a reality (even bringing Windows, I repeat WINDOWS, into the fold) with C++11 (and the subsequent releases) and it's, in my humble opinion, utter folly to try and change the way code will fundamentally compile depending on the distribution you run. If Debian cares, make a proposal to the C++ committee, bribe^H convince members to see their way (or threaten^H blackmail^H show them the dangers of continuing down the road they're on). Heck, fork C/C++ and call it E or C+++ or c-safe or something - or more reasonably - write a tool to convert C to rust or D-without-the-standard-library and announce only tools in that/those languages will be allowed in the standard distribution. But for Heaven's sake, please don't try to redefine C.
- anon4 11y ago
  Or they'll force the compiler writers' hand. See: EGCS.
- nix0n 11y ago
  If you read the thread more thoroughly, Ian Jackson is actually more on the side of Debian leaving things the way they are upstream, while others are proposing that Debian enable Link Time Optimization for the sake of performance.
- maaku 11y ago
  Isn't that what Ian said in TFA?
kyberias 11y ago
Glossary: UB = Undefined behavior LTO = Link time optimization
vmorgulis 11y ago
ubsan could help a lot: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
- Kristine1975 11y ago
  And ASAN (the Address Sanitizer). I always have them turned on for development builds (slowdown of about 100%). Only release builds have them switched off. Recently: Use-after-free of a memory block. ASAN told me: 1. Where and in which thread the use-after-free occurred (including stacktrace). 2. Where and in which thread the memory was deallocated (including stacktrace). 3. Where and in which thread the memory had originally been allocated (including stacktrace). Fixing the bug took about 30s with that info.
jbb555 11y ago
I agree with this wholeheartedly. I can see the C language being forked into the one we are getting and the one users actually want.
zokier 11y ago
I'm getting bit annoyed about these sorts whiny posts that float up occasionally. Here is a simple three step program if you don't like what C language is today: 1) Write your own damn spec with no "problematic" UB. Hookers and blackjack are optional 2) Write a compiler for your shiny new language. 3) Start writing/porting code to your new language There you have it, UB problem solved once and for all. I'm not sold over on Regehrs work, but at least he is doing something with his Friendly-C proposal. This has the added benefit that we get something more concrete to discuss and debate about instead of vague "optimizations breaking my code are bad". It is completely unfeasible to try to turn back time on C and somehow magically make compilers deduce programmer intent from some random crap that you throw at them. Like it or not, computers are based on rules, and standards are the best way we have to establish those rules among large number of parties.
- Kristine1975 11y ago
  Designing your own language isn't even necessary. There's Rust, Ada, Modula, Pascal... if you're feeling masochistic, use COBOL ;-)
  - dalke 11y ago
    The Wikipedia page for COBOL list several places where programming constructs lead to undefined behavior.
- kbenson 11y ago
  The problem is not just with "what C language is today", but with the fact that the use of undefined behavior allows future optimizations to possibly change how your code that was written today, tomorrow, or years ago, compiles and runs in the future.
TorKlingberg 11y ago
There has been very little new undefined behavior added to C recently. The only one I can think of is INT_MIN % -1, but most people seem to think that was a good idea. All compilers were already treating it as UB, even though it wasn't. http://blog.regehr.org/archives/175 http://blog.regehr.org/archives/175
StringyBob 11y ago
I'm a digital hardware (chips) guy. We use compilers (synthesis tools) and normally won't get a second chance to recompile if the logic gates in our silicon chip are wrong as a result of compiler bugs or misinterpretation of source. We automatically distrust the compiler (synthesis tool) to do the right thing. You formally prove the 'compiled' output (logic gates) that will be manufactured matches with the source code of the design (verilog/vhdl) using tools written independently to the compiler. This isn't easy, and I know the problem space is larger, but does anyone ever do this for software?
- nkurz 11y ago
  SQLite is usually (and correctly) held as an example of thorough software testing: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html Perhaps you could offer a sense of how this compares to the hardware testing practices you use?
  - whaaswijk 11y ago
    Typically in hardware design you'd use combinational equivalence checking (CEC) to formally prove that a synthesized design is correct. See https://en.wikipedia.org/wiki/Formal_equivalence_checking https://en.wikipedia.org/wiki/Formal_equivalence_checking. In CEC you use a SAT solver to prove that a new (synthesized and optimized) circuit is equivalent to some reference model (aka golden model) which you know is correct. So instead of relying on just testing (which is incomplete) you have a formal proof that your design is correct. Of course you still have to trust that the SAT solver works correctly... :-)
  - StringyBob 11y ago
    In hardware design you typically test functionality through logic simulation or emulation (effectively running the code in a computer simulation or fpga), use test harnesses, look for code coverage, run unit tests, random code fuzzing, code assertions etc. You might also do formal checks for some assertions to e.g. avoid deadlocks. A secondary check is that the source that you functionally tested is logically equivalent to what you manufacture. This is where you are not checking your code, but the issue is trust of compiler/ compiler optimisations in synthesis. It needs to be redone if you recompile - that's the step I don't really ever see in software development - if I use a different compiler option or underlying instruction set architecture to the SQLite Dev team, do I still trust my binary? Of course the level of paranoia is far higher in hardware where it costs multiple millions of dollars to crank out a new spin of a chip!
ArkyBeagle 11y ago
So the real argument is whether LTO is on by default? Because if it's the source of trouble and it's off by default, then there's only trouble when you choose to use it.
deleted 11y ago
[deleted]
pgeorgi 11y ago
For coreboot, I started making gcc (we use our own toolchain for various reasons already) emit some link-time error inducing symbol, because their current approach (__builtin_trap()) is nice for undefined behavior in userspace programs that can segfault, but not for firmware that silently hangs. https://review.coreboot.org/14364 https://review.coreboot.org/14364
alexeiz 11y ago
On the contrary, I'm currently implementing LTO in our project because it allows to uncover and fix the whole slew of latent problems related to ODR violations. At the same time I'm enabling UB sanitizers, so I'm confident that LTO will actually make our code more robust and reliable. The usefulness of LTO goes beyond better optimization, and it should not be ignored.

118 points by Sindisil 11y ago

"But sadly it seems that the notion that our most basic and widely-used programming language should be one that's fit for programming in is not yet fully accepted." Ouch. I usually hear that about C++, not C. =[

twoodfin 11y ago
My guess is that many of the optimizations causing increasing trouble were introduced to improve C++ code performance in pursuit of the "zero-cost abstraction" goal. But many are just as applicable to C (and may even be below the level that a compiler like gcc or clang knows the difference).
- ArkyBeagle 11y ago
  The whole "zero-cost abstraction" thing puzzles me. Abstraction is a means to an end, not an end unto itself.
  - rbancroft 11y ago
    We want abstractions to make things easier to do, but we don't want to pay for them with CPU cycles... that's what a zero-cost abstraction means. It's not about adding in useless abstractions. The ideal is software that is faster to write and easier to understand and maintain, while offering uncompromised execution performance.

haberman 11y ago

I can see both sides of this. But looking at this argument: > There are two ways to evaluate the the C specification's rightness and properness. [...] The second is to ask what is most useful. And there again the C committee have clearly failed. Just two days on HN we saw this article: https://news.ycombinator.com/item?id=11468603 https://news.ycombinator.com/item?id=11468603 In which it says: > no matter the kind of software, performance is almost always worse than our customers would like it to be. That is why all of this is happening. There is a market demand for performance. Compilers that increasingly exploit UB in C is just a manifestation of that market demand.

michaelt 11y ago
There is a market demand for performance. You could fulfil that demand without breaking existing code by making the riskier optimisations configurable and turned off by default.
- webkike 11y ago
  They are usually. Most people don't change the -03 in their build files after a compiler update though.
  - ktRolster 11y ago
    -O4 exists for 'risky' optimizations. -O3 is for optimizations that are mostly not risky. -O2 is for optimizations that don't involve a speed/size tradeoff -O1 is for optimizations that don't slow your compiler down too much Obviously these are fuzzy categories, but -O4 is where "undefined behavior" optimizations belong. Alternately, each individual optimization has its own flag, and can be turned on individually.
    - haberman 11y ago
      I've never heard of -O4 for "risky" optimizations, and I can't find any reference to this. Do you have one?
      - ktRolster 11y ago
        If you search for "gcc O4" you'll find some references (it's tricky because if you search for -O4 Google will search for everything without O4). Currently, there are no optimizations that fit in that category, but as recently as 2003 I remember seeing it in the man pages. I believe there was -O5 and -O6 too, but I don't trust my memory enough to be certain.
        haberman 11y ago
        The results for that query all seem to say that "gcc -O4" doesn't exist. There is one proposal from Chris Lattner for adding this to Clang, but it's not clear that this ever happened.

deleted 11y ago

[deleted]

Glyptodon 11y ago

UB being the various "undefined behavior" bits of C?

cpeterso 11y ago
Yes. Compiler developers like it; everyone hates it. :)
- twoodfin 11y ago
  You like it when aliasing rules allow your compiler to copy the target of a pointer into a register, rather than read it from memory over and over in case an intervening write through another pointer could have overwritten it.

hinkley 11y ago

I have to say, strictly as an uninformed outsider, this bit of nastiness is just pushing Go, Rust and even Swift higher on my priorities list. I don't think I'm the only one.

ktRolster 11y ago
Well then, don't look too deeply at the nastiness in Go, Rust, and Swift.
- dsfuoi 11y ago
  Can you elaborate? Hopefully they don't have straight UB do they?
  - ktRolster 11y ago
    Hopefully they don't have straight UB do they? Yes, most languages do, unless they are formally defined (ML is formally defined, but most other languages are not. In this book http://www.amazon.com/Masterminds-Programming-Conversations-Creators-Languages/dp/0596515170 http://www.amazon.com/Masterminds-Programming-Conversations-... several of the language creators say that fully defining the language formally is not worth the effort).
  - zokier 11y ago
    Well, Rust for example does not have a formal well-defined behavior to start with, so its kinda fuzzy about UB too. And of course `unsafe` is another story altogether.
    - mastax 11y ago
      Can you elaborate? The rust docs [1] seem to disagree: Unlike C, Undefined Behavior is pretty limited in scope in Rust. All the core language cares about is preventing the following things: - Dereferencing null or dangling pointers - Reading uninitialized memory - Breaking the pointer aliasing rules - Producing invalid primitive values: - dangling/null references - a bool that isn't 0 or 1 - an undefined enum discriminant - a char outside the ranges [0x0, 0xD7FF] and [0xE000, 0x10FFFF] - A non-utf8 str - Unwinding into another language - Causing a data race And all of those things require `unsafe`, so safe rust cannot do any of them (barring compiler or unsafe library or OS bugs). Edit: And I must admit I don't know much about language theory or formal definition, but there is also a self-described formal grammar [2]. [1]: https://doc.rust-lang.org/nomicon/races.html https://doc.rust-lang.org/nomicon/races.html [2]: https://doc.rust-lang.org/grammar.html https://doc.rust-lang.org/grammar.html

jabl 11y ago

The obvious problem with the "The second is to ask what is most useful" approach is that if you ask 1000 C programmers, 990 are not clueful enough to give any sane answer and will pollute your answer database with noise, and of the remaining 10, 5 are language lawyers and favor approach #1 anyway, and the other 5 will give 20 different answers (Note: above numbers completely made up). So then the compiler developers give up and implement stuff 1) according to the standard document 2) such that performance on SPEC cpu is maximized.

_delirium 11y ago
A related comment: http://blog.regehr.org/archives/1287 http://blog.regehr.org/archives/1287
- jabl 11y ago
  Indeed. The only realistic way out of this conundrum AFAICS is some kind of consensus around "friendly C"/"boring C" or such that compilers then can implement, whether it's an annex to the official C standard or some other kind of spec. That, or then everybody switches to Rust. :) (hey, I can daydream can I?).
  - pcwalton 11y ago
    And everyone will have to accept the performance loss--and there will be performance loss. This is acceptable for many applications, don't get me wrong. But it will be a tough sell. If you want to get people using a Friendly C, you need to start convincing the biggest users of C and C++ that they should give up performance for a simpler dialect of the language. Like politicians responding to voters, compiler authors respond to what their customers demand. Up to now, their customers have demanded performance. It's not their fault for listening to them.
    - dsfuoi 11y ago
      Performance of real code, not micro-benchmarks, won't change. And this can work both ways. Friendly C would almost certainly forbid non-compatible type pointer castings, which would actually enable additional optimizations.

ktRolster 11y ago

This narrative of `fault' has two very serious problems. That can be said almost any time people are trying to find fault. Better to just look for a solution, and not worry about whose fault it is.

cpeterso 11y ago

What are the particular compiler behaviors or optimizations they would disable?

twoodfin 11y ago
A quick look at the first example provided suggests the restriction on comparing pointers to internal members of distinct structures.
SQLite 11y ago
How about allowing pointers to be compared against NULL after they have been passed to free(): "free(x); if( x!=NULL ){...}" That was completely harmless (and a useful idiom) for 45 years, and now suddenly it is flagged as UB and has to be changed. Why? Wouldn't it be great if "memset(0,p,0)" was a harmless no-op? It was for time out of mind. But no more. For bonus points: Can we have a #pragma that tells the compiler to abort with an error if the target machine uses any representation for signed integers other than twos-complement?
- dsfuoi 11y ago
  Why? Because programmers learned that it is correct, even though it wasn't.(Or at least it stopped being, in C99 an on.) Here is a car analogy. It is like learning to drive and thinking that you are allowed to drive over a yellow-turning-to-red light. However the rules say, you should stop if you are physically able to. In reality almost everyone tries to get over than yellow. On a rare occasion they get spotted and pay the fine. C strives for maximum performance, it will not check things for you if you don't ask it to. Having a library function that performs those checks for everyone, would go against that rule. Why would someone else have to pay the performance penalty for you? Write a wrapper or a macro that performs the check, couple of lines, it is that easy. If you strive to write portable C code it should work regardless of signed representation. C defines all the range macros for types, and gives you types that guarantee certain ranges. Assuming you use those types and macros, I'm really curious what incantations, that couldn't been solved in a portable manner, require you to know the signed representation.
- 11y ago

haberman 11y ago

One area that worries me that I never see anyone talk about is LTO between C and C++. How do you even reason about that? I asked a C++ and compiler expert about this once and he told me: "I believe both GCC's and LLVM's LTO will happily cross this barrier, so it doesn't offer you any real protection from their optimizers." There is no single standard that defines what UB is for a mixture of C and C++. There are clearly some parts of C++ that are trying to improve interoperability with C, like "standard layout" classes. The best you can do is try to simultaneously follow the rules for both languages when you mix the two.

Kristine1975 11y ago
Why not flag each function with "C" or "C++" depending on where they originate from? The linker can then perform only optimizations that are valid for the respective language.
DSMan195276 11y ago
Personally, I think this is blown out of proportion a bit - There are things to complain about, but LTO problems are really only caused by other issues. But with that said, what you described isn't really a worry: LTO is done on the compilers internal representation of the code (IE. GIMPLE or LLVM IR). This representation is generated based on the rules of each language, and optimizations are performed on this representation instead of the original source representation. Both C and C++ (and anything else) are converted into these representations. LTO simply keeps the GIMPLE or LLVM IR around until link time, and then when the program is linked optimizations are performed over the entire representation of the program. Crossing the language barrier shouldn't be a problem, because the GIMPLE has it's own rules to follow to make sure everything still functions the same. Once you reach this point both languages are already compiled in the practical sense, they're just not actual machine code yet. I would expect however, that because of the differences there are far less optimization opportunities to be taken advantage of.
- haberman 11y ago
  Ok let's look at a specific example then. // foo.c #include <stdlib.h> typedef struct { int x; } s; s *make_s() { s *ret = malloc(sizeof(*ret)); ret->x = 5; return ret; } void free_s(s *val) { free(val); } // bar.cc // This class is standard-layout and matches "s" from C. class C { int getX() { return x_; } private: int x_; } extern C* make_s(); extern void free_s(C* c); int main(void) { C* c = make_s(); int ret = c->getX(); free_s(c); return ret; } C++ says you can only call a method on an object whose lifetime has begun. Can C begin the lifetime of a C++ object?

dsfuoi 11y ago

C is designed to be capable of running of vastly different architectures, allowing the programmer to use that flexibility when it actually isn't needed, or is harmful. I think the C coding mindset should be this: if an approach requires code that isn't unambiguously defined, change the approach. If this means more boring coding, to circumvent cute tricks, so be it. If you need that extra 5% speed, use assembly instead of bending C.

astrobe_ 11y ago
The problem is that people often don't even know what is UB and what is not. People don't even always know what is portable and what isn't, to begin with. Some even seem to believe that "it's written in C, so it's portable".
- dsfuoi 11y ago
  I strongly agree. There is a serious lack of respect of the C rules even among those who teach and this is passed onto their students. It is really hard to convince them to change ways after that. A couple months ago, as a curiosity, I watched a few videos in a series on programming in C in Windows environment. The teacher was a serious programmer, but the first thing that went out the window was strict aliasing. After that assumptions of integer sizes and range started creeping in. It came apparent that the teacher knew C, but only superficially, signedness, integer promotions and usual arithmetic conversions were treated like a nuisance. If the code compiled and ran, it was good. Those videos were the first C programming experience for at least several hundred people.

sneak 11y ago

Amazing that in 2016, the best programmers in the world have not chosen to solve the "display mailing list archives on a webpage without terrible line wrapping" problem yet.

oldmanjay 11y ago
It was hard to think about, it should be hard to read

robertelder 11y ago

The issue of creeping optimizations causing problems far in the future is one of the reasons that I suggested using unsigned intergers in C as much as possible: "Compiler authors will likely support this as long as possible, but the peer pressure of needing things to go faster and faster will likely push them to exploit more and more undefined behaviour to their advantage in the future. Their argument will be: 'After all, who has sympathy for those who don't follow the standard?'." From: http://blog.robertelder.org/signed-or-unsigned-part-2/ http://blog.robertelder.org/signed-or-unsigned-part-2/ Lots of people disagree with me, but its nice to have less UB issues to worry about far in the future.

ComputerGuru 11y ago

I don't know if it's because I convinced myself a long time ago that I'll never be smart enough to write correct C/C++ code (though that hasn't (yet) stopped it from being my primary development language) or if it's because I just because I'm a secret masochist, but I disagree with this proposal/suggestion. Regardless of who is right and who is wrong in this matter, I think if everyone took a step back we could at least agree that it makes absolutely no sense to fix this on a (single Linux) distribution level. For Debian to configure/patch compilers on their platform to "narrow" undefined behavior is insane and ineffectual. Software isn't "validated" on/against a particular OS, it's validated on a compiler basis. Breaking this assumption introduces a massive schism. While Debian is an amazing distro with plenty of clout (I'm a FreeBSD guy, but Debian comes second), it's terrifying to imagine a new generation of "cross-platform" C/C++ software that can only be verified working on Debian (or with Debian's fork/re-configured compiler). We've come so close to making truly cross-platfrom C++ code a reality (even bringing Windows, I repeat WINDOWS, into the fold) with C++11 (and the subsequent releases) and it's, in my humble opinion, utter folly to try and change the way code will fundamentally compile depending on the distribution you run. If Debian cares, make a proposal to the C++ committee, bribe^H convince members to see their way (or threaten^H blackmail^H show them the dangers of continuing down the road they're on). Heck, fork C/C++ and call it E or C+++ or c-safe or something - or more reasonably - write a tool to convert C to rust or D-without-the-standard-library and announce only tools in that/those languages will be allowed in the standard distribution. But for Heaven's sake, please don't try to redefine C.

anon4 11y ago
Or they'll force the compiler writers' hand. See: EGCS.
nix0n 11y ago
If you read the thread more thoroughly, Ian Jackson is actually more on the side of Debian leaving things the way they are upstream, while others are proposing that Debian enable Link Time Optimization for the sake of performance.
maaku 11y ago
Isn't that what Ian said in TFA?

kyberias 11y ago

Glossary: UB = Undefined behavior LTO = Link time optimization

vmorgulis 11y ago

ubsan could help a lot: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

Kristine1975 11y ago
And ASAN (the Address Sanitizer). I always have them turned on for development builds (slowdown of about 100%). Only release builds have them switched off. Recently: Use-after-free of a memory block. ASAN told me: 1. Where and in which thread the use-after-free occurred (including stacktrace). 2. Where and in which thread the memory was deallocated (including stacktrace). 3. Where and in which thread the memory had originally been allocated (including stacktrace). Fixing the bug took about 30s with that info.

jbb555 11y ago

I agree with this wholeheartedly. I can see the C language being forked into the one we are getting and the one users actually want.

zokier 11y ago

I'm getting bit annoyed about these sorts whiny posts that float up occasionally. Here is a simple three step program if you don't like what C language is today: 1) Write your own damn spec with no "problematic" UB. Hookers and blackjack are optional 2) Write a compiler for your shiny new language. 3) Start writing/porting code to your new language There you have it, UB problem solved once and for all. I'm not sold over on Regehrs work, but at least he is doing something with his Friendly-C proposal. This has the added benefit that we get something more concrete to discuss and debate about instead of vague "optimizations breaking my code are bad". It is completely unfeasible to try to turn back time on C and somehow magically make compilers deduce programmer intent from some random crap that you throw at them. Like it or not, computers are based on rules, and standards are the best way we have to establish those rules among large number of parties.

Kristine1975 11y ago
Designing your own language isn't even necessary. There's Rust, Ada, Modula, Pascal... if you're feeling masochistic, use COBOL ;-)
- dalke 11y ago
  The Wikipedia page for COBOL list several places where programming constructs lead to undefined behavior.
kbenson 11y ago
The problem is not just with "what C language is today", but with the fact that the use of undefined behavior allows future optimizations to possibly change how your code that was written today, tomorrow, or years ago, compiles and runs in the future.

TorKlingberg 11y ago

There has been very little new undefined behavior added to C recently. The only one I can think of is INT_MIN % -1, but most people seem to think that was a good idea. All compilers were already treating it as UB, even though it wasn't. http://blog.regehr.org/archives/175 http://blog.regehr.org/archives/175

StringyBob 11y ago

I'm a digital hardware (chips) guy. We use compilers (synthesis tools) and normally won't get a second chance to recompile if the logic gates in our silicon chip are wrong as a result of compiler bugs or misinterpretation of source. We automatically distrust the compiler (synthesis tool) to do the right thing. You formally prove the 'compiled' output (logic gates) that will be manufactured matches with the source code of the design (verilog/vhdl) using tools written independently to the compiler. This isn't easy, and I know the problem space is larger, but does anyone ever do this for software?

nkurz 11y ago
SQLite is usually (and correctly) held as an example of thorough software testing: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html Perhaps you could offer a sense of how this compares to the hardware testing practices you use?
- whaaswijk 11y ago
  Typically in hardware design you'd use combinational equivalence checking (CEC) to formally prove that a synthesized design is correct. See https://en.wikipedia.org/wiki/Formal_equivalence_checking https://en.wikipedia.org/wiki/Formal_equivalence_checking. In CEC you use a SAT solver to prove that a new (synthesized and optimized) circuit is equivalent to some reference model (aka golden model) which you know is correct. So instead of relying on just testing (which is incomplete) you have a formal proof that your design is correct. Of course you still have to trust that the SAT solver works correctly... :-)
- StringyBob 11y ago
  In hardware design you typically test functionality through logic simulation or emulation (effectively running the code in a computer simulation or fpga), use test harnesses, look for code coverage, run unit tests, random code fuzzing, code assertions etc. You might also do formal checks for some assertions to e.g. avoid deadlocks. A secondary check is that the source that you functionally tested is logically equivalent to what you manufacture. This is where you are not checking your code, but the issue is trust of compiler/ compiler optimisations in synthesis. It needs to be redone if you recompile - that's the step I don't really ever see in software development - if I use a different compiler option or underlying instruction set architecture to the SQLite Dev team, do I still trust my binary? Of course the level of paranoia is far higher in hardware where it costs multiple millions of dollars to crank out a new spin of a chip!

ArkyBeagle 11y ago

So the real argument is whether LTO is on by default? Because if it's the source of trouble and it's off by default, then there's only trouble when you choose to use it.

deleted 11y ago

[deleted]

pgeorgi 11y ago

For coreboot, I started making gcc (we use our own toolchain for various reasons already) emit some link-time error inducing symbol, because their current approach (__builtin_trap()) is nice for undefined behavior in userspace programs that can segfault, but not for firmware that silently hangs. https://review.coreboot.org/14364 https://review.coreboot.org/14364

alexeiz 11y ago

On the contrary, I'm currently implementing LTO in our project because it allows to uncover and fix the whole slew of latent problems related to ODR violations. At the same time I'm enabling UB sanitizers, so I'm confident that LTO will actually make our code more robust and reliable. The usefulness of LTO goes beyond better optimization, and it should not be ignored.