7 ms·

The standard committee’s position is that if undefined behavior isn’t easy for the programmer to detect, why would it be easy for the compiler to detect? I’m a

The standard committee’s position is that if undefined behavior isn’t easy for the programmer to detect, why would it be easy for the compiler to detect? I’m a little more familiar with the C++ committee than the C committee. The C++ committee prefers to declare something an error than to declare it undefined behavior. They only declare something undefined when they believe it would be unreasonably difficult for the compiler to detect the problem (e.g., using multiple, incompatible, definitions for an inline function; which can happen if you have a macro expanding differently in different parts of the code, or you have multiple definitions for the function, but only one definition is ever visible to the compiler at any moment in time). I’m pretty sure the “signed overflow is undefined” rule is something of a special case: it should be easy to detect when source code doesn’t have a hard coded upper bound, but giving an error or warning in all cases will create too many false positives, and declaring that it wraps on overflow has been deemed unacceptable by the committee.

GoblinSlayer 5y ago
UB problems happen only when compilers detect it, understand it is an opportunity for optimization, because they are allowed to do anything, then do that optimization. When compiler can't detect UB, it actually works as expected: when integers overflow they do exactly that and nothing else, when null pointers are dereferenced they do exactly that and nothing else, when uninitialized memory is read it does exactly that and nothing else.
- foxfluff 5y ago
  That's not true at all. Compilers can detect some instances of UB, and they will happily warn when they do. Most instances of UB are ones that the compiler could not detect. Your compiler isn't going to detect that your integer overflows, it assumes that it won't. Shit blows up if that assumption was wrong. If you want to detect UB, you should run ubsan and the like. The compiler is not running it.
  - GoblinSlayer 5y ago
    It blows up, but not as violently as when the compiler adds fuel to it.
  - jerf 5y ago
    In some sense, the problem isn't that the compiler "can't" detect that an integer addition will overflow. The problem is that with some exceptions (a loop that is clearly indexing along an array, for example), the compiler will have to flag every integer addition as UB. UB notifications aren't that useful if you're getting a dozen for every five line function in your entire codebase. Programmers do not, in general, want to do the amount of work it takes to safely add two numbers together. You need something "weird". Like an error path for every addition. Or a type system for ranges, where you actually have to specify ranges (because adding two bytes together that you casually left as 0-255 will create a new ranged value 0-510; multiplication is even more fun). Or some other exotic idea. So we just let them overflow and the chips fall where they may.
    - foxfluff 5y ago
      > In some sense, the problem isn't that the compiler "can't" detect that an integer addition will overflow. It absolutely can't if it doesn't know what the values are going to be. > the compiler will have to flag every integer addition as UB No, I disagree. If it has no knowledge of my runtime values, it can't flag addition as UB because it isn't. It's UB only if I'm using values that would overflow, and the compiler in general can't know that. If I've done my program right, it will never overflow. There is no UB, any flag would be just absolutely wrong. The compiler couldn't detect UB. There's nothing to flag. If it knows the values, then of course it can detect it and flag it: x.c:3:18: warning: integer overflow in expression of type ‘int’ results in ‘-2’ [-Woverflow] 3 | int a=2147483647+2147483647; > Programmers do not, in general, want to do the amount of work it takes to safely add two numbers together. I agree that overflow-checking is needlessly sucky in C but that's not "the problem." I usually make sure my arithmetic is safe but the compiler won't know it.
- MauranKilom 5y ago
  > UB problems happen only when compilers detect it, understand it is an opportunity for optimization, because they are allowed to do anything, then do that optimization. This is a deep misconception, and not at all how most undefined behavior is related to optimization. Trivial example: Compilers assume that your variables aren't written to randomly from other threads. Without this assumption, virtually no optimization would be possible. Therefore, data races are UB - they violate a hard assumption of the compiler. But at no point did the compiler say "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". This is the same for e.g. removed overflow checks. Compilers can optimize many loops only if they assume signed integer overflow never happens. So, because compilers should be able to assume this, it is UB if it happens in your code. The same logic that deduces "this loop cannot wrap around" deduces "this if condition [an overflow check] cannot ever be true". But it's easier for a programmer who got their UB-reliant checks optimized out to attribute malice to the compiler than to understand optimization fundamentals, and thus we get people complaining to high heaven and back.
  - GoblinSlayer 5y ago
    >But at no point did the compiler say "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". That's because the compiler doesn't detect the situation. If it knew, it could, say, remove all code leading to that point - it's legal and the resulting code would be faster. >Compilers can optimize many loops only if they assume signed integer overflow never happens. I don't think this happens. Most loops iterate over arrays, and at least in C tradition unsigned integers are used as array index with no signed integers in sight.
  - dooglius 5y ago
    You are strawmanning the parent, he did not claim that compilers will go "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". Your description of the compiler's behavior matches his description; the compilers detect a case where the program might exhibit behavior defined ISO C spec (signed integer overflow), see it as a potential for optimization because they are allowed to do anything (certain passes can be made if overflow is assumed not to happen), then do that optimization.
- steveklabnik 5y ago
  UB is closer to "we assume this can never happen and make decisions accordingly" than "Ha ha, I caught you making a mistake, let's actively try to mess up your life now."
  - GoblinSlayer 5y ago
    They know they mess up, they just have defense for it.
- klodolph 5y ago
  Reading uninitialized memory is not UB... but anyway. Here's an example of how compilers treat null dereferencing as UB: void func(struct x *aptr, struct x *bptr) { x->a = 5; y->a = 10; x->a = 10; } This is a contrived example but you can imagine writing code that makes a sequence of assignments like this. An optimizing compiler can remove the "dead store" which assigns 5 to x->a. However, this optimization is valid because writing to a null pointer is undefined... if writing to a null pointer trapped (SIGFAULT), which is what it actually does at runtime, then the optimization would be incorrect... because you could observe the intermediate value of x->a = 5. The compiler is not really "detecting that your program has undefined behavior". Instead, it is assuming that undefined behavior doesn't happen, and optimizes your program using that assumption. It's unclear what "detecting undefined behavior" would mean here... what kind of diagnostic the compiler should emit, or how to suppress the diagnostic. It's clear that this simple code is pervasive, and the preferred approach for dealing with it is somewhat outside the scope of a normal C compiler... something like static analysis, runtime instrumentation, or some kind of formal methods.
  - foxfluff 5y ago
    > However, this optimization is valid because writing to a null pointer is undefined... if writing to a null pointer trapped (SIGFAULT), which is what it actually does at runtime, then the optimization would be incorrect... because you could observe the intermediate value of x->a = 5. I'm not sure this is a good example, because there's no way for you to read them from the same thread (and reading from another thread would be a race). Reading them from a signal handler would yield unspecified values (unless you're using lock-free atomic types, in which case we're probably not worried about optimizing these assignments). > When the processing of the abstract machine is interrupted by receipt of a signal, the values of objects that are neither lock-free atomic objects nor of type volatile sig_atomic_t are unspecified, as is the state of the floating-point environment.
    - klodolph 5y ago
      That's a good point, however I think it's not hard to come up with a different example that shows how compilers assume that pointers aren't NULL. Like, void function(struct x *ptr) { ptr->x = 1; if (ptr == NULL) { some_big_chunk_of_code(); } } The idea is that this could be the result of inlining, where the inlined code does a null check, but in the context it's being inlined into, we already know ptr is not null.
- chc 5y ago
  The problem is that this doesn't mean anything, at least in terms of the C programming language. You say "When integers overflow, they do that and nothing else," but there is no definition of signed integer overflow, so there is no "that" for them to do. That is what it means for something to be undefined behavior. You have some definition in your mind, which might happen to match what some versions of some compilers do at some optimization levels — but that isn't actually the definition, because there isn't one.
  - GoblinSlayer 5y ago
    Wrapping arithmetic is supported by gcc and clang.
deleted 5y ago
[deleted]

GoblinSlayer 5y ago

UB problems happen only when compilers detect it, understand it is an opportunity for optimization, because they are allowed to do anything, then do that optimization. When compiler can't detect UB, it actually works as expected: when integers overflow they do exactly that and nothing else, when null pointers are dereferenced they do exactly that and nothing else, when uninitialized memory is read it does exactly that and nothing else.

foxfluff 5y ago
That's not true at all. Compilers can detect some instances of UB, and they will happily warn when they do. Most instances of UB are ones that the compiler could not detect. Your compiler isn't going to detect that your integer overflows, it assumes that it won't. Shit blows up if that assumption was wrong. If you want to detect UB, you should run ubsan and the like. The compiler is not running it.
- GoblinSlayer 5y ago
  It blows up, but not as violently as when the compiler adds fuel to it.
- jerf 5y ago
  In some sense, the problem isn't that the compiler "can't" detect that an integer addition will overflow. The problem is that with some exceptions (a loop that is clearly indexing along an array, for example), the compiler will have to flag every integer addition as UB. UB notifications aren't that useful if you're getting a dozen for every five line function in your entire codebase. Programmers do not, in general, want to do the amount of work it takes to safely add two numbers together. You need something "weird". Like an error path for every addition. Or a type system for ranges, where you actually have to specify ranges (because adding two bytes together that you casually left as 0-255 will create a new ranged value 0-510; multiplication is even more fun). Or some other exotic idea. So we just let them overflow and the chips fall where they may.
  - foxfluff 5y ago
    > In some sense, the problem isn't that the compiler "can't" detect that an integer addition will overflow. It absolutely can't if it doesn't know what the values are going to be. > the compiler will have to flag every integer addition as UB No, I disagree. If it has no knowledge of my runtime values, it can't flag addition as UB because it isn't. It's UB only if I'm using values that would overflow, and the compiler in general can't know that. If I've done my program right, it will never overflow. There is no UB, any flag would be just absolutely wrong. The compiler couldn't detect UB. There's nothing to flag. If it knows the values, then of course it can detect it and flag it: x.c:3:18: warning: integer overflow in expression of type ‘int’ results in ‘-2’ [-Woverflow] 3 | int a=2147483647+2147483647; > Programmers do not, in general, want to do the amount of work it takes to safely add two numbers together. I agree that overflow-checking is needlessly sucky in C but that's not "the problem." I usually make sure my arithmetic is safe but the compiler won't know it.
MauranKilom 5y ago
> UB problems happen only when compilers detect it, understand it is an opportunity for optimization, because they are allowed to do anything, then do that optimization. This is a deep misconception, and not at all how most undefined behavior is related to optimization. Trivial example: Compilers assume that your variables aren't written to randomly from other threads. Without this assumption, virtually no optimization would be possible. Therefore, data races are UB - they violate a hard assumption of the compiler. But at no point did the compiler say "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". This is the same for e.g. removed overflow checks. Compilers can optimize many loops only if they assume signed integer overflow never happens. So, because compilers should be able to assume this, it is UB if it happens in your code. The same logic that deduces "this loop cannot wrap around" deduces "this if condition [an overflow check] cannot ever be true". But it's easier for a programmer who got their UB-reliant checks optimized out to attribute malice to the compiler than to understand optimization fundamentals, and thus we get people complaining to high heaven and back.
- GoblinSlayer 5y ago
  >But at no point did the compiler say "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". That's because the compiler doesn't detect the situation. If it knew, it could, say, remove all code leading to that point - it's legal and the resulting code would be faster. >Compilers can optimize many loops only if they assume signed integer overflow never happens. I don't think this happens. Most loops iterate over arrays, and at least in C tradition unsigned integers are used as array index with no signed integers in sight.
- dooglius 5y ago
  You are strawmanning the parent, he did not claim that compilers will go "oh, you wrote to this variable without synchronization! Now I'll show you, hehehe!". Your description of the compiler's behavior matches his description; the compilers detect a case where the program might exhibit behavior defined ISO C spec (signed integer overflow), see it as a potential for optimization because they are allowed to do anything (certain passes can be made if overflow is assumed not to happen), then do that optimization.
steveklabnik 5y ago
UB is closer to "we assume this can never happen and make decisions accordingly" than "Ha ha, I caught you making a mistake, let's actively try to mess up your life now."
- GoblinSlayer 5y ago
  They know they mess up, they just have defense for it.
klodolph 5y ago
Reading uninitialized memory is not UB... but anyway. Here's an example of how compilers treat null dereferencing as UB: void func(struct x *aptr, struct x *bptr) { x->a = 5; y->a = 10; x->a = 10; } This is a contrived example but you can imagine writing code that makes a sequence of assignments like this. An optimizing compiler can remove the "dead store" which assigns 5 to x->a. However, this optimization is valid because writing to a null pointer is undefined... if writing to a null pointer trapped (SIGFAULT), which is what it actually does at runtime, then the optimization would be incorrect... because you could observe the intermediate value of x->a = 5. The compiler is not really "detecting that your program has undefined behavior". Instead, it is assuming that undefined behavior doesn't happen, and optimizes your program using that assumption. It's unclear what "detecting undefined behavior" would mean here... what kind of diagnostic the compiler should emit, or how to suppress the diagnostic. It's clear that this simple code is pervasive, and the preferred approach for dealing with it is somewhat outside the scope of a normal C compiler... something like static analysis, runtime instrumentation, or some kind of formal methods.
- foxfluff 5y ago
  > However, this optimization is valid because writing to a null pointer is undefined... if writing to a null pointer trapped (SIGFAULT), which is what it actually does at runtime, then the optimization would be incorrect... because you could observe the intermediate value of x->a = 5. I'm not sure this is a good example, because there's no way for you to read them from the same thread (and reading from another thread would be a race). Reading them from a signal handler would yield unspecified values (unless you're using lock-free atomic types, in which case we're probably not worried about optimizing these assignments). > When the processing of the abstract machine is interrupted by receipt of a signal, the values of objects that are neither lock-free atomic objects nor of type volatile sig_atomic_t are unspecified, as is the state of the floating-point environment.
  - klodolph 5y ago
    That's a good point, however I think it's not hard to come up with a different example that shows how compilers assume that pointers aren't NULL. Like, void function(struct x *ptr) { ptr->x = 1; if (ptr == NULL) { some_big_chunk_of_code(); } } The idea is that this could be the result of inlining, where the inlined code does a null check, but in the context it's being inlined into, we already know ptr is not null.
chc 5y ago
The problem is that this doesn't mean anything, at least in terms of the C programming language. You say "When integers overflow, they do that and nothing else," but there is no definition of signed integer overflow, so there is no "that" for them to do. That is what it means for something to be undefined behavior. You have some definition in your mind, which might happen to match what some versions of some compilers do at some optimization levels — but that isn't actually the definition, because there isn't one.
- GoblinSlayer 5y ago
  Wrapping arithmetic is supported by gcc and clang.

deleted 5y ago

[deleted]