5 ms·

C programmers should be able to expect that "optimizations" will not transform program meaning. And because C is so low level, certain types of optimizations ma

C programmers should be able to expect that "optimizations" will not transform program meaning. And because C is so low level, certain types of optimizations may be more difficult or impossible. If the pointer was explicitly set to NULL, the compiler can justifiably deduce the branch will not be taken but the deduction "if the programmer dereferenced the pointer it must not be NULL" is not based on a sound rule. In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky. Optimization should always be secondary to correctness.

msbarnett 10y ago
> C programmers should be able to expect that "optimizations" will not transform program meaning. If x is null, the program in #2 has no meaning. The only way to preserve its meaning is to assume x is not null. > Optimization should always be secondary to correctness. If x is null, the program in #2 has no correctness. The only way to salvage its correctness is to assume x is not null.
- vyodaiken 10y ago
  Exactly. You are assuming that a poorly written standard is correct and engineering practice of working programs is incorrect.
  - msbarnett 10y ago
    Ok, so you want compilers to generate a translation for program #2 that works "correctly" to your mind when x is null. Please explain to the class the meaning of the construct int y = *x; when x is null, so that all conforming C compilers can be updated to generate code for this case "correctly".
    - yorwba 10y ago
      Something like assert(x); int y = *x; is probably closer to the intended meaning. Checking if(!x) afterwards is still dead code, but at least the program is guaranteed to fail in a defined manner. Of course, if implemented this way, every dereference would have to be paired with an assert call, bringing the performance down to the level of Java. (While bringing the memory safety up to the level of Java.)
      - msbarnett 10y ago
        That's the opposite of what he wants. He wants a compiler that produces a translation which derefrences the null and follows the true branch. He wants null to equal a valid addressable address, which is completely nonstandard and not portable to everywhere C is used, to be standard that portable compilers emit code for. He wants to imagine that null and zero are the same thing.
        vyodaiken 10y ago
        Nobody wants that.
      - to3m 10y ago
        If the code for program #2 isn't enough to describe the desired behaviour, perhaps it isn't expressible in terms of standard C. Here's what "x86-64 clang 3.9.1" gives for foo with -O3: (https://godbolt.org/g/0xe1OD https://godbolt.org/g/0xe1OD) foo(int*): # @foo(int*) test rdi, rdi je .LBB0_1 jmp bar() # TAILCALL .LBB0_1: ret (You might like to compare this with the article's claims.) More the sort of thing that I'd expect to be generated (since it's a more accurate rendering of the C code): foo(int*): # @foo(int*) mov eax, [rdi] test rdi, rdi je .LBB0_1 jmp bar() # TAILCALL .LBB0_1: ret I know that NULL is a valid address on x64 (it's zero), and on any system that doesn't completely suck it will be unmapped. (If I'm using one of the ones that sucks, I'll already be prepared for this. But luckily I'm not.) So I'd like to feel confident that the compiler will just leave the dereferences in - that way, when I make some horrid mistake, I'll at least get an error at runtime. But rather than compile my mistakes as written, it seems that the compiler would prefer to double down on them.
        scatters 10y ago
        If you want the compiler to leave the dereferences in, use `-fno-delete-null-pointer-checks` or ask your compiler vendor for an equivalent option. Compilers delete null pointer checks by default (on typical platforms) because it's what most users want.
  - deleted 10y ago
    [deleted]
maxlybbert 10y ago
> C programmers should be able to expect that "optimizations" will not transform program meaning. That's the official rule, but it's "program meaning as defined by the standard." It's not perfect, but nobody's come up with a better alternative. We get bugs because programmers expect some meaning that's not in the standard. But compilers are written according to the standard, not according to some folklore about what reasonable or experienced programmers expect. * Again, the idea isn't that the compiler found a mistake and will do its best to make you regret it. Derefencing a pointer is a clear statement that the programmer believes the pointer isn't NULL. The standard allows the compiler to believe that statement. Partly because the language doesn't define what to do if the statement is false.
- nwmcsween 10y ago
  > But compilers are written according to the standard. Written to the writers _interpretation_ of the standard. I bet money that every compiler written from a text standard hasn't followed said standard. It would be nice if a standard included code fragments used to show/test the validity of what is stated.
  - adrianN 10y ago
    There are examples in the standard.
- vyodaiken 10y ago
  Actually that's not correct. The standard says the behavior is up to the compiler. The compiler author took that as a license to produce a non truth preserving transformation of the code. The actual current clang behavior also satisfies the standard as written.
  - maxlybbert 10y ago
    > The standard says the behavior is up to the compiler. I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key. Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it. Undefined behavior means that the program is invalid, but the compiler isn't expected to notice the error. Whatever the compiler spits out is acceptable by definition. The compiler can generate a program that doesn't follow the rules of C; or that only does something weird when undefined behavior is triggered, but the weird behavior doesn't take place on the same line as the undefined behavior; etc. It's certainly true that "the compiler isn't expected to notice the error" doesn't prohibit a compiler from noticing the error. A compiler can notice, but it's standard conforming even if it doesn't. I should probably mention that when I say "the standard" I mean the C language standard; the POSIX standard may add requirements such that something undefined according to the language is well defined on POSIX.
mattkrause 10y ago
> In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky. That's the way it's often explained but it's not really what happens--the compiler doesn't scan for undefined behavior and then replace it with random operations. Instead, it's applying a series of transformations that preserve the program's semantics if the program stays "in bounds", avoiding invoking undefined behaviors. I agree that equating "if the programmer dereferenced the pointer" and "the pointer must not be NULL" betrays a....touching naivety about the quality of a lot of code, but if you start from the premise that the programmer shouldn't be doing that, the results aren't totally insane.
- vyodaiken 10y ago
  "touching naivete"is a good way of putting it.

msbarnett 10y ago

> C programmers should be able to expect that "optimizations" will not transform program meaning. If x is null, the program in #2 has no meaning. The only way to preserve its meaning is to assume x is not null. > Optimization should always be secondary to correctness. If x is null, the program in #2 has no correctness. The only way to salvage its correctness is to assume x is not null.

vyodaiken 10y ago
Exactly. You are assuming that a poorly written standard is correct and engineering practice of working programs is incorrect.
- msbarnett 10y ago
  Ok, so you want compilers to generate a translation for program #2 that works "correctly" to your mind when x is null. Please explain to the class the meaning of the construct int y = *x; when x is null, so that all conforming C compilers can be updated to generate code for this case "correctly".
  - yorwba 10y ago
    Something like assert(x); int y = *x; is probably closer to the intended meaning. Checking if(!x) afterwards is still dead code, but at least the program is guaranteed to fail in a defined manner. Of course, if implemented this way, every dereference would have to be paired with an assert call, bringing the performance down to the level of Java. (While bringing the memory safety up to the level of Java.)
    - msbarnett 10y ago
      That's the opposite of what he wants. He wants a compiler that produces a translation which derefrences the null and follows the true branch. He wants null to equal a valid addressable address, which is completely nonstandard and not portable to everywhere C is used, to be standard that portable compilers emit code for. He wants to imagine that null and zero are the same thing.
      - vyodaiken 10y ago
        Nobody wants that.
    - to3m 10y ago
      If the code for program #2 isn't enough to describe the desired behaviour, perhaps it isn't expressible in terms of standard C. Here's what "x86-64 clang 3.9.1" gives for foo with -O3: (https://godbolt.org/g/0xe1OD https://godbolt.org/g/0xe1OD) foo(int*): # @foo(int*) test rdi, rdi je .LBB0_1 jmp bar() # TAILCALL .LBB0_1: ret (You might like to compare this with the article's claims.) More the sort of thing that I'd expect to be generated (since it's a more accurate rendering of the C code): foo(int*): # @foo(int*) mov eax, [rdi] test rdi, rdi je .LBB0_1 jmp bar() # TAILCALL .LBB0_1: ret I know that NULL is a valid address on x64 (it's zero), and on any system that doesn't completely suck it will be unmapped. (If I'm using one of the ones that sucks, I'll already be prepared for this. But luckily I'm not.) So I'd like to feel confident that the compiler will just leave the dereferences in - that way, when I make some horrid mistake, I'll at least get an error at runtime. But rather than compile my mistakes as written, it seems that the compiler would prefer to double down on them.
      - scatters 10y ago
        If you want the compiler to leave the dereferences in, use `-fno-delete-null-pointer-checks` or ask your compiler vendor for an equivalent option. Compilers delete null pointer checks by default (on typical platforms) because it's what most users want.
- deleted 10y ago
  [deleted]

maxlybbert 10y ago

> C programmers should be able to expect that "optimizations" will not transform program meaning. That's the official rule, but it's "program meaning as defined by the standard." It's not perfect, but nobody's come up with a better alternative. We get bugs because programmers expect some meaning that's not in the standard. But compilers are written according to the standard, not according to some folklore about what reasonable or experienced programmers expect. * Again, the idea isn't that the compiler found a mistake and will do its best to make you regret it. Derefencing a pointer is a clear statement that the programmer believes the pointer isn't NULL. The standard allows the compiler to believe that statement. Partly because the language doesn't define what to do if the statement is false.

nwmcsween 10y ago
> But compilers are written according to the standard. Written to the writers _interpretation_ of the standard. I bet money that every compiler written from a text standard hasn't followed said standard. It would be nice if a standard included code fragments used to show/test the validity of what is stated.
- adrianN 10y ago
  There are examples in the standard.
vyodaiken 10y ago
Actually that's not correct. The standard says the behavior is up to the compiler. The compiler author took that as a license to produce a non truth preserving transformation of the code. The actual current clang behavior also satisfies the standard as written.
- maxlybbert 10y ago
  > The standard says the behavior is up to the compiler. I think this statement is correct, but it's the kind of thing people say when they confuse implementation defined behavior and undefined behavior. And that distinction is key. Implementation defined behavior means the compiler gets to choose what it will do, document the choice, and then stick to it. Undefined behavior means that the program is invalid, but the compiler isn't expected to notice the error. Whatever the compiler spits out is acceptable by definition. The compiler can generate a program that doesn't follow the rules of C; or that only does something weird when undefined behavior is triggered, but the weird behavior doesn't take place on the same line as the undefined behavior; etc. It's certainly true that "the compiler isn't expected to notice the error" doesn't prohibit a compiler from noticing the error. A compiler can notice, but it's standard conforming even if it doesn't. I should probably mention that when I say "the standard" I mean the C language standard; the POSIX standard may add requirements such that something undefined according to the language is well defined on POSIX.

mattkrause 10y ago

> In fact, the whole concept that the compiler can make any transformation it wants in the presence of UB is wacky. That's the way it's often explained but it's not really what happens--the compiler doesn't scan for undefined behavior and then replace it with random operations. Instead, it's applying a series of transformations that preserve the program's semantics if the program stays "in bounds", avoiding invoking undefined behaviors. I agree that equating "if the programmer dereferenced the pointer" and "the pointer must not be NULL" betrays a....touching naivety about the quality of a lot of code, but if you start from the premise that the programmer shouldn't be doing that, the results aren't totally insane.

vyodaiken 10y ago
"touching naivete"is a good way of putting it.