10 ms·

For people claiming that this is a compiler bug, it really is not. I am no expert, but what I understood is that the C standard defines some situations (such a

For people claiming that this is a compiler bug, it really is not. I am no expert, but what I understood is that the C standard defines some situations (such as an overflow error) which result in "undefined behaviour". In this situation, the compiler is free to do whatever he wants. This is in fact what happens here. This is clearly annoying to programmers. In this case, it is even hard to avoid undefined behaviour even if the source of undefined behaviour (a pointer overflow) is known. Why is this done? This way the compiler can do some aggresive optimizations which are only valid if there is no undefined behaviour (e.g. no overflow, null pointer acces...). There are various good articles about this. To make the situation more complex for programmers, it is possible to write programs which exhibit undefined behaviour but work fine in practice. Until the compiler tries to do a specific optimization. This is basically why you shouldn't use C. Long-time C programmers know a list of operations which are undefined behaviour, and are usually able to point out a few of these in a program written by newbie (ironically, these programs may run and pass tests just fine).

frederikvs 9y ago
To be specific, only certain types of overflow are undefined behaviour. Signed integers and pointers have undefined behaviour on overflow. Unsigned integers on the other hand are defined to wrap around to 0. For those interested, good places to start learning more about this are John Regehr's blog [0], or the llvm blog [1]. [0] http://blog.regehr.org/archives/213 http://blog.regehr.org/archives/213 [1] http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
- cperciva 9y ago
  Technically unsigned integer wrapping isn't considered to be "overflow".
  - cperciva 9y ago
    Replying to myself because it's too late to edit the comment: I'm not sure why people are voting that down. Section 6.2.5 paragraph 9 of the C99 standard states that: A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.
cperciva 9y ago
C is unusual in having an explicit concept of undefined behaviour. But that doesn't mean that other languages don't have undefined behaviour... it just means that nobody has gone through the standard (if there even is a standard beyond "the language does what this implementation produces") and identified all of the gaps.
- johncolanduoni 9y ago
  Is it that unusual? I've seen undefined behavior be specifically mentioned in both the Java and CLR standards. Rust's (admittedly squishy) "reference" also specifies a lot of behavior as undefined explicitly (along with a general "if you break mandated invariants behavior is undefined").
  - steveklabnik 9y ago
    In Rust's case, this is limited to unsafe code, at least; safe code should not have UB. (Though, IIRC there is one case right now, but that's a bug) UB is basically "breaking the invariants safe code expects." Overflow is an example of something that's not UB in Rust, for example. (It's a "program error" and well-defined as two's compliment wrapping.)
masklinn 9y ago
> For people claiming that this is a compiler bug, it really is not. In fact TFA specifically points that out multiple times.
- mpweiher 9y ago
  Claims don't become true by repetition.
GTP 9y ago
I'm no expert too, but please read this opinion about the problem [0]. I think he has a good point saying that undefined behavior != the compiler has the right to do wathewer it wants. [0] http://article.gmane.org/gmane.os.plan9.general/76989 http://article.gmane.org/gmane.os.plan9.general/76989
- MaulingMonkey 9y ago
  I'd agree with "undefined behavior != the compiler should do whatever it wants" - but I feel I must admit that the standard lets it, that many compiler authors apparently disagree with me, and that even those who do are sympathetic to my point of view or agree with me may have a point when they say that reason about programmer intention is a difficult bag of worms - and that preserving such, while maintaining "critical" optimizations, may not be as easy as it sounds. There's always -O0.
  - mpweiher 9y ago
    There is a difference between "complies with the standard" and "is bug free". There were C compilers before there was a C standard. They had bugs.
    - MaulingMonkey 9y ago
      > There is a difference between "complies with the standard" and "is bug free". You're not wrong - but the dividing line between bug and feature request is subjective, and just how much (meaningful) difference between the two there is depends on the authors of the compiler. If you compile with -fno-strict-aliasing and the optimizer breaks your code based solely on strict aliasing violations anyways, by all means, report that as a bug. If you use a compiler that has no -fno-strict-aliasing equivalent, by all means, switch to a better compiler when they WONTFIX your feature request.
  - GTP 9y ago
    Of course if the compiler has a certain behavior I have to deal with it, I just feel that certain optimizations based on undefined behavior are too aggressive, later this evening I will maybe write in more detail about this in anoter comment below.
Asooka 9y ago
Maybe. I really want to be writing code for a 2-complement integer CPU with 64-bit registers and a unified memory model, where addresses are unsigned integers and address 0 is NULL. I really, really want a portable-assembly flavour of C which would be machine target-dependent and would e.g. throw an error when compiling a pointer comparison for a machine with a nonuniform memory model, if the pointers cannot be proven to be part of the same object. The current model where parts of my code can just get deleted SILENTLY is really hard to reason about. At least add an -Werror-deleted-code (or -Werror-tautology for code that's tautological due to UB), so I can be alerted if that optimisation is done.
- jjnoakes 9y ago
  I think if you had a way to enable warnings about deleted code you would be flooded with warnings from macro expansions, function inlining, template instantiations, and other valid and desirable optimizations.
- caf 9y ago
  I don't think it's that hard to reason about. Checking for overflow (and pointer-out-of-object-bounds, which is a special case of overflow) in C is like defusing a bomb: you can't cut the red wire and then see if it exploded, you have to check that it won't explode before you cut the wire.
infimum 9y ago
does anyone know, how hard it would be to have a compiler flag -Wundefined that tells me when i write code with undefined behaviour? this way i can fix these issues myself and don't have to rely on the compiler inferring my intentions.
- kutkloon7 9y ago
  GCC has -fsanitize=undefined. I don't think it detects all kinds of undefined behavior (I haven't tried it myself). You should be able to find more information if you please. Edit: mannykannot noted somewhere else that clang has the same flag.
  - staticassertion 9y ago
    These are runtime sanitizers designed to aid testing and fuzzing, not meant for running in production software. It involves instrumentation.
naasking 9y ago
> I am no expert, but what I understood is that the C standard defines some situations (such as an overflow error) which result in "undefined behaviour". In this situation, the compiler is free to do whatever he wants. Except the proper answer to this scenario is to refuse to compile with an appropriate error, not to eliminate your code.
- deleted 9y ago
  [deleted]
- klodolph 9y ago
  I think there's a misunderstanding here—the undefined behavior occurs at runtime, not at compile time, so rejecting it outright at compile time is not a sensible solution (it's not sensible to reject something if you don't even know that it happens). The modern purpose of undefined behavior is to enable these kinds of optimizations anyway, so I can write: int *x = some_call(); ... *x = 5; ... if (x == NULL) abort(); The compiler will optimize out the abort(), which is a good optimization since obviously x can't be NULL otherwise *x = 5; would be wrong. You might say that the programmer should be responsible for removing the abort()—but it might be inlined from a different function or something like that. Rejecting the program with an error would seriously piss me off. We want our compiler to be aggressive about optimization under the assumption that if we write clean, standards-conformant code, the compiler will optimize out any bounds checks that it can prove will pass, so we can be free to put bounds checks everywhere and get good performance. That's hard, and it's not obvious that it's hard, and people routinely underestimate how hard it is to write standards-conformant C. There are a lot of alternative ideas in the pipeline but we don't automatically get to blame the compiler vendor, even though sometimes it really has been the compiler's fault. People get rather riled up about undefined behavior. There is a good subset of people who rather like fast C code (optimizing out paths with UB can make code much faster), and a good subset of people who think C should just be safe and predictable. If you want safe and predictable, C is a tough sell.
  - kbenson 9y ago
    Arguing the extremes of this as people tend to do about this is not constructive. A compiler flag that warned or errored on branches that were optimized out but the branch included an abort or exit within a certain distance (AST nodes?) of the branch would be useful. Possibly even disallowing optimizing our that conditional that branch if it can be determined it contains any sort of exit or abort before returning control flow.

frederikvs 9y ago

To be specific, only certain types of overflow are undefined behaviour. Signed integers and pointers have undefined behaviour on overflow. Unsigned integers on the other hand are defined to wrap around to 0. For those interested, good places to start learning more about this are John Regehr's blog [0], or the llvm blog [1]. [0] http://blog.regehr.org/archives/213 http://blog.regehr.org/archives/213 [1] http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

cperciva 9y ago
Technically unsigned integer wrapping isn't considered to be "overflow".
- cperciva 9y ago
  Replying to myself because it's too late to edit the comment: I'm not sure why people are voting that down. Section 6.2.5 paragraph 9 of the C99 standard states that: A computation involving unsigned operands can never overflow, because a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type.

cperciva 9y ago

C is unusual in having an explicit concept of undefined behaviour. But that doesn't mean that other languages don't have undefined behaviour... it just means that nobody has gone through the standard (if there even is a standard beyond "the language does what this implementation produces") and identified all of the gaps.

johncolanduoni 9y ago
Is it that unusual? I've seen undefined behavior be specifically mentioned in both the Java and CLR standards. Rust's (admittedly squishy) "reference" also specifies a lot of behavior as undefined explicitly (along with a general "if you break mandated invariants behavior is undefined").
- steveklabnik 9y ago
  In Rust's case, this is limited to unsafe code, at least; safe code should not have UB. (Though, IIRC there is one case right now, but that's a bug) UB is basically "breaking the invariants safe code expects." Overflow is an example of something that's not UB in Rust, for example. (It's a "program error" and well-defined as two's compliment wrapping.)

masklinn 9y ago

> For people claiming that this is a compiler bug, it really is not. In fact TFA specifically points that out multiple times.

mpweiher 9y ago
Claims don't become true by repetition.

GTP 9y ago

I'm no expert too, but please read this opinion about the problem [0]. I think he has a good point saying that undefined behavior != the compiler has the right to do wathewer it wants. [0] http://article.gmane.org/gmane.os.plan9.general/76989 http://article.gmane.org/gmane.os.plan9.general/76989

MaulingMonkey 9y ago
I'd agree with "undefined behavior != the compiler should do whatever it wants" - but I feel I must admit that the standard lets it, that many compiler authors apparently disagree with me, and that even those who do are sympathetic to my point of view or agree with me may have a point when they say that reason about programmer intention is a difficult bag of worms - and that preserving such, while maintaining "critical" optimizations, may not be as easy as it sounds. There's always -O0.
- mpweiher 9y ago
  There is a difference between "complies with the standard" and "is bug free". There were C compilers before there was a C standard. They had bugs.
  - MaulingMonkey 9y ago
    > There is a difference between "complies with the standard" and "is bug free". You're not wrong - but the dividing line between bug and feature request is subjective, and just how much (meaningful) difference between the two there is depends on the authors of the compiler. If you compile with -fno-strict-aliasing and the optimizer breaks your code based solely on strict aliasing violations anyways, by all means, report that as a bug. If you use a compiler that has no -fno-strict-aliasing equivalent, by all means, switch to a better compiler when they WONTFIX your feature request.
- GTP 9y ago
  Of course if the compiler has a certain behavior I have to deal with it, I just feel that certain optimizations based on undefined behavior are too aggressive, later this evening I will maybe write in more detail about this in anoter comment below.

Asooka 9y ago

Maybe. I really want to be writing code for a 2-complement integer CPU with 64-bit registers and a unified memory model, where addresses are unsigned integers and address 0 is NULL. I really, really want a portable-assembly flavour of C which would be machine target-dependent and would e.g. throw an error when compiling a pointer comparison for a machine with a nonuniform memory model, if the pointers cannot be proven to be part of the same object. The current model where parts of my code can just get deleted SILENTLY is really hard to reason about. At least add an -Werror-deleted-code (or -Werror-tautology for code that's tautological due to UB), so I can be alerted if that optimisation is done.

jjnoakes 9y ago
I think if you had a way to enable warnings about deleted code you would be flooded with warnings from macro expansions, function inlining, template instantiations, and other valid and desirable optimizations.
caf 9y ago
I don't think it's that hard to reason about. Checking for overflow (and pointer-out-of-object-bounds, which is a special case of overflow) in C is like defusing a bomb: you can't cut the red wire and then see if it exploded, you have to check that it won't explode before you cut the wire.

infimum 9y ago

does anyone know, how hard it would be to have a compiler flag -Wundefined that tells me when i write code with undefined behaviour? this way i can fix these issues myself and don't have to rely on the compiler inferring my intentions.

kutkloon7 9y ago
GCC has -fsanitize=undefined. I don't think it detects all kinds of undefined behavior (I haven't tried it myself). You should be able to find more information if you please. Edit: mannykannot noted somewhere else that clang has the same flag.
- staticassertion 9y ago
  These are runtime sanitizers designed to aid testing and fuzzing, not meant for running in production software. It involves instrumentation.

naasking 9y ago

> I am no expert, but what I understood is that the C standard defines some situations (such as an overflow error) which result in "undefined behaviour". In this situation, the compiler is free to do whatever he wants. Except the proper answer to this scenario is to refuse to compile with an appropriate error, not to eliminate your code.

deleted 9y ago
[deleted]
klodolph 9y ago
I think there's a misunderstanding here—the undefined behavior occurs at runtime, not at compile time, so rejecting it outright at compile time is not a sensible solution (it's not sensible to reject something if you don't even know that it happens). The modern purpose of undefined behavior is to enable these kinds of optimizations anyway, so I can write: int *x = some_call(); ... *x = 5; ... if (x == NULL) abort(); The compiler will optimize out the abort(), which is a good optimization since obviously x can't be NULL otherwise *x = 5; would be wrong. You might say that the programmer should be responsible for removing the abort()—but it might be inlined from a different function or something like that. Rejecting the program with an error would seriously piss me off. We want our compiler to be aggressive about optimization under the assumption that if we write clean, standards-conformant code, the compiler will optimize out any bounds checks that it can prove will pass, so we can be free to put bounds checks everywhere and get good performance. That's hard, and it's not obvious that it's hard, and people routinely underestimate how hard it is to write standards-conformant C. There are a lot of alternative ideas in the pipeline but we don't automatically get to blame the compiler vendor, even though sometimes it really has been the compiler's fault. People get rather riled up about undefined behavior. There is a good subset of people who rather like fast C code (optimizing out paths with UB can make code much faster), and a good subset of people who think C should just be safe and predictable. If you want safe and predictable, C is a tough sell.
- kbenson 9y ago
  Arguing the extremes of this as people tend to do about this is not constructive. A compiler flag that warned or errored on branches that were optimized out but the branch included an abort or exit within a certain distance (AST nodes?) of the branch would be useful. Possibly even disallowing optimizing our that conditional that branch if it can be determined it contains any sort of exit or abort before returning control flow.