4 ms·

Despite this, tools will often not warn of undefined behavior or these premature optimizations. Quite the opposite, the compilers are almost user hostile for tr

Despite this, tools will often not warn of undefined behavior or these premature optimizations. Quite the opposite, the compilers are almost user hostile for trying to ensure safety and correctness. I cannot see into the minds of the GCC and LLVM teams responsible for the C frontends, but to the extent that you'd think the compiler should do anything here, they seem to think it's largely not their job to improve safety or correctness. That's the job of static analysis tools (which can cost a lot of money), runtime analysis tools (which can cost a lot of time in writing tests that exercise all code paths), and fuzzers (yet more time). Punting all of this extremely important reasoning to "the programmer" is ridiculous when it's been proven over and over again that even incredibly smart C programmers get this wrong.

saagarjha 7y ago
Compilers could certainly do a better job when performing optimizations that “abuse” undefined behavior, but in general this isn’t a trivial problem. The way optimization passes are set up can often make it difficult to generate warnings that have a low false-positive rate.
- AaronFriel 7y ago
  Do you think it might be a problem that undefined behavior optimizations and/or pattern matches are so prolific in C that reducing the "false-positive rate" of ... actually bad undefined behavior (?) would be the hard thing? Why not disallow undefined behavior and instead have the compiler emit: "Hey, this branch is never taken, if you want us not to compile it remove the dead code". Seems like that's exactly the same sort of putting the onus on the programmer that andrepd mentioned, but without the side effect of millions of device drivers are zero day remote code execution vulnerabilities waiting to happen?
  - cyphar 7y ago
    The array access example shows why this isn't generally possible -- the programmer hasn't written any bounds-checking code or assertions (they just wrote x[y]). Should the compiler emit bounds-checks? If yes, your code is now slower but there isn't an easy way to disable this "feature" (if x is heap-allocated or passed as a pointers, how will the compiler know what kind of bounds check is sufficient?). If no, you have undefined behaviour and crashes. Even Rust made a mistake with restricting undefined behaviour -- they accidentally allowed creating references to fields in packed structures (resulting in possibly unaligned references which is undefined behaviour in Rust) in safe code. These days you get a warning telling you to use unsafe or copy the value, but they have yet to make it an error. For comparison, C also gives you a warning for the same problem (or an error with -Werror). I also want to point out that modern compilers have different sanitisers (-fsanitize=...) which cause your program to abort() on memory unsafety or other violations. It's effectively a better valgrind which is compiled into your program.
    - AaronFriel 7y ago
      I would argue that most of the time, yes, the compiler should emit bounds checks and only in performance sensitive code should they be removed (or where the compiler can prove the bounds check holds for a range). I don't know how anyone who has studied the history of vulnerabilities and security issues and code would think this is a hard tradeoff: > If yes, your code is now slower but there isn't an easy way to disable this "feature" (if x is heap-allocated or passed as a pointers, how will the compiler know what kind of bounds check is sufficient?). > If no, you have undefined behaviour and crashes. The latter choice has resulted in how many millions or billions in losses, perhaps even lives lost?
      - cyphar 7y ago
        The problem is that if you want to permit users to do things like dereference pointers and do pointer arithmetic (a requirement for a C-like language) there is no way for the compiler to ensure (even at runtime, because the pointer might not be to memory managed by the language) that the pointer dereference is valid. Okay, maybe you can add a small overhead to array accesses (which is what Rust did), but you can't stop undefined behaviour entirely. Rust has plenty of undefined behaviour (most can only be triggered through unsafe -- which is the whole argument behind the language -- but as I mentioned there are some cases where even they made a mistake and made an operation safe when it should've have been).
        AaronFriel 7y ago
        The benefit of "unsafe" blocks is that the surface area to audit is dramatically reduced, so that yes, if you want high performance pointer mangling and arithmetic, you can do it, but your code will require more attention. In C/C++, undefined behavior and unsafe operations abound and it is nigh impossible to isolate what is "important" to review from a security standpoint.
    - saagarjha 7y ago
      Address Sanitizer is actually a worse Valgrind, because it misses certain bugs and makes up for it by being faster.
  - DSMan195276 7y ago
    > Why not disallow undefined behavior and instead have the compiler emit ... You can't "disallow" all undefined behavior in C without fundamentally changing the language to something different (Which would also likely make it useless for it's intended use-case). Some of the details related to undefined behavior are simply not possible for the compiler to detect without extra information, hence why they are undefined. `Rust` has the same thing, you can invoke undefined behavior anywhere in the program through use of `unsafe` anywhere else. And Ex. How is the compiler supposed to determine if a particular function is ever called with a `NULL` parameter? How does it know if a particular signed addition may overflow? How does it know if two parameters to a function alias in an invalid way? How does it know if a particular pointer lacks a `const` and is actually backed by read-only memory? And most importantly, how does it determine all of this at compile time? With that not all undefined behavior-based optimizations are unexpected or warrant a warning or error (In-fact, I would say at the very least the majority of them don't). Undefined behavior is much more complicated then just dead-code removal. Even then though, you don't always want dead-code removal warned about - Imagine getting a warning about every inline function or macro that results in some dead-code. To be clear, I'm not saying there aren't obvious problems that have come from this approach, and I think some of the choices for undefined-behavior are clear mistakes now even though they made some sense in the 80s or 90s (And some of those can usually be turned off, thankfully). And I think there are also some additions that could be made to the language that could solve a lot of the existing problems (Like non-null pointers, or fat pointers, though I'm not exactly holding my breath...), but the existence of undefined-behavior is not in-and-of-itself the actual problem with the language, and "disallowing" it outright is simply not possible. I would actually wager undefined-behavior exists in some form in a lot more languages than you'd expect, just less documented. Rust certainly still has it, despite it's numerous safety-focused features. And even if you add things like fat pointers to C that would clearly reduce possible out-of-bounds mistakes, the language would still need to provide some way to create one from a size and a pointer, and that still allows for undefined-behavior cases to happen - my point being, while adding such a thing is still clearly better, it does not actually remove the undefined-behavior possibility.

saagarjha 7y ago

Compilers could certainly do a better job when performing optimizations that “abuse” undefined behavior, but in general this isn’t a trivial problem. The way optimization passes are set up can often make it difficult to generate warnings that have a low false-positive rate.

AaronFriel 7y ago
Do you think it might be a problem that undefined behavior optimizations and/or pattern matches are so prolific in C that reducing the "false-positive rate" of ... actually bad undefined behavior (?) would be the hard thing? Why not disallow undefined behavior and instead have the compiler emit: "Hey, this branch is never taken, if you want us not to compile it remove the dead code". Seems like that's exactly the same sort of putting the onus on the programmer that andrepd mentioned, but without the side effect of millions of device drivers are zero day remote code execution vulnerabilities waiting to happen?
- cyphar 7y ago
  The array access example shows why this isn't generally possible -- the programmer hasn't written any bounds-checking code or assertions (they just wrote x[y]). Should the compiler emit bounds-checks? If yes, your code is now slower but there isn't an easy way to disable this "feature" (if x is heap-allocated or passed as a pointers, how will the compiler know what kind of bounds check is sufficient?). If no, you have undefined behaviour and crashes. Even Rust made a mistake with restricting undefined behaviour -- they accidentally allowed creating references to fields in packed structures (resulting in possibly unaligned references which is undefined behaviour in Rust) in safe code. These days you get a warning telling you to use unsafe or copy the value, but they have yet to make it an error. For comparison, C also gives you a warning for the same problem (or an error with -Werror). I also want to point out that modern compilers have different sanitisers (-fsanitize=...) which cause your program to abort() on memory unsafety or other violations. It's effectively a better valgrind which is compiled into your program.
  - AaronFriel 7y ago
    I would argue that most of the time, yes, the compiler should emit bounds checks and only in performance sensitive code should they be removed (or where the compiler can prove the bounds check holds for a range). I don't know how anyone who has studied the history of vulnerabilities and security issues and code would think this is a hard tradeoff: > If yes, your code is now slower but there isn't an easy way to disable this "feature" (if x is heap-allocated or passed as a pointers, how will the compiler know what kind of bounds check is sufficient?). > If no, you have undefined behaviour and crashes. The latter choice has resulted in how many millions or billions in losses, perhaps even lives lost?
    - cyphar 7y ago
      The problem is that if you want to permit users to do things like dereference pointers and do pointer arithmetic (a requirement for a C-like language) there is no way for the compiler to ensure (even at runtime, because the pointer might not be to memory managed by the language) that the pointer dereference is valid. Okay, maybe you can add a small overhead to array accesses (which is what Rust did), but you can't stop undefined behaviour entirely. Rust has plenty of undefined behaviour (most can only be triggered through unsafe -- which is the whole argument behind the language -- but as I mentioned there are some cases where even they made a mistake and made an operation safe when it should've have been).
      - AaronFriel 7y ago
        The benefit of "unsafe" blocks is that the surface area to audit is dramatically reduced, so that yes, if you want high performance pointer mangling and arithmetic, you can do it, but your code will require more attention. In C/C++, undefined behavior and unsafe operations abound and it is nigh impossible to isolate what is "important" to review from a security standpoint.
  - saagarjha 7y ago
    Address Sanitizer is actually a worse Valgrind, because it misses certain bugs and makes up for it by being faster.
- DSMan195276 7y ago
  > Why not disallow undefined behavior and instead have the compiler emit ... You can't "disallow" all undefined behavior in C without fundamentally changing the language to something different (Which would also likely make it useless for it's intended use-case). Some of the details related to undefined behavior are simply not possible for the compiler to detect without extra information, hence why they are undefined. `Rust` has the same thing, you can invoke undefined behavior anywhere in the program through use of `unsafe` anywhere else. And Ex. How is the compiler supposed to determine if a particular function is ever called with a `NULL` parameter? How does it know if a particular signed addition may overflow? How does it know if two parameters to a function alias in an invalid way? How does it know if a particular pointer lacks a `const` and is actually backed by read-only memory? And most importantly, how does it determine all of this at compile time? With that not all undefined behavior-based optimizations are unexpected or warrant a warning or error (In-fact, I would say at the very least the majority of them don't). Undefined behavior is much more complicated then just dead-code removal. Even then though, you don't always want dead-code removal warned about - Imagine getting a warning about every inline function or macro that results in some dead-code. To be clear, I'm not saying there aren't obvious problems that have come from this approach, and I think some of the choices for undefined-behavior are clear mistakes now even though they made some sense in the 80s or 90s (And some of those can usually be turned off, thankfully). And I think there are also some additions that could be made to the language that could solve a lot of the existing problems (Like non-null pointers, or fat pointers, though I'm not exactly holding my breath...), but the existence of undefined-behavior is not in-and-of-itself the actual problem with the language, and "disallowing" it outright is simply not possible. I would actually wager undefined-behavior exists in some form in a lot more languages than you'd expect, just less documented. Rust certainly still has it, despite it's numerous safety-focused features. And even if you add things like fat pointers to C that would clearly reduce possible out-of-bounds mistakes, the language would still need to provide some way to create one from a size and a pointer, and that still allows for undefined-behavior cases to happen - my point being, while adding such a thing is still clearly better, it does not actually remove the undefined-behavior possibility.