26 ms·

The problem has always been that in practice any nontrivial codebase had UB somewhere (invalidating the entire program!) and diagnosing any particular instance

The problem has always been that in practice any nontrivial codebase had UB somewhere (invalidating the entire program!) and diagnosing any particular instance was generally painful until recently. Compilers didn't point most things out, sanitizers didn't exist, and prior to 2011, I don't think there was even a list of UB in C besides the entire standard. C++ is still largely in that position AFAIK. It's a complete disaster on all sides.

fooker 5y ago
Invalidating the entire program is theoretically correct, but not really a useful statement. In practice, weird miscompilations due to UB are just slightly more difficult to debug than your usual segfault. You can generally keep reducing your problem to localize the issue in the code. Also, such issues are not very common because the value obtained from an UB operation is usually nonsense (shifting past bitwidth, out of bounds array element, etc) so a compiler switching things around is just garbage-in-garbage-out. It is of course a serious issue if a program is actually depending on such a value for a crucial operation. That's how you get exploits, with or without the compiler doing something clever.
- anonymoushn 5y ago
  One of my least favorite bugs came from a loop that iterated from *p to *(p+n) (exclusive). For n=0 and values of p other than null, the loop body never executes. For n=0 and p=null, the loop body is allowed to execute, because null+0 is allowed to be any value.
  - fooker 5y ago
    Nice example! Seems solvable by printf debugging though..
    - taneq 5y ago
      The fact that it’s fixable doesn’t mean it’s fun…
- barrkel 5y ago
  Miscompilations due to UB are normally silent and can remove checks written into the code for security purposes; e.g. checks for overflowing signed integers, or for null pointers. They're not harder to debug. Debugging them isn't the issue. The problem is knowing that the code in the editor doesn't correspond to the code under execution.
- vlovich123 5y ago
  All I can do is point out that very big names in CS (including Linus) disagree with you: http://www.yodaiken.com/wp-content/uploads/2018/05/ub-1.pdf http://www.yodaiken.com/wp-content/uploads/2018/05/ub-1.pdf This is probably because they’ve seen the effects of the foot gun first hand. They also recall what C/C++ looked like before the standards bodies made exploiting UB in compilers open season. There’s also a reason why Rust doesn’t have any UB in its safe dialect even though it sits on infrastructure capable of most/all the same possible optimizations via LLVM. There was in fact a direct kernel security issue UB exploitation caused in the Linux kernel whereas without that optimization there is no security issue. if I recall correctly the code looked something like: Some_value = ptr->value If (!ptr) { return; } This worked correctly in all cases before the compiler added the optimization. It worked less well after because the !ptr check was UB since it followed a dereference. It also wasn’t immediately obvious this code was broken because there was no diagnostic nor any indication that upgrading a compiler would suddenly elide the check. The value-add of such optimizations at scale vs the correctness issues exploiting it causes is questionable. The problem wasn’t that it’s not fixable. It’s that it takes time to find and you may not even find it until after it’s being exploited.
  - UncleMeat 5y ago
    That code is buggy if ptr can be null. There isn't an especially rigorous distinction between a bug and a vuln. So saying "this was definitely a safe bug and the compiler elevated it to a vuln" can be a thing in very specific situations, but it also isn't generalizable in any way. Linus is an important figure, but there are plenty of world experts on the other side of this argument. And there is even a third side that states that not only should the compiler avoid optimizations based on UB assumptions, C implementations should instead function more like a virtual machine for a PDP-11 and have everything under the sun be defined (at great cost of performance). If you take an especially pessimistic view from the compiler, then really really basic stuff is almost impossible. A write to a dereferenced pointer is almost equivalent to a full program havoc. This invalidates almost all facts that the compiler knows and prevents virtually every optimization near that write. Heck, technically this can interfere with things like vtables and invalidate any sort of devirtualization, which is hugely valuable for performance.

fooker 5y ago

Invalidating the entire program is theoretically correct, but not really a useful statement. In practice, weird miscompilations due to UB are just slightly more difficult to debug than your usual segfault. You can generally keep reducing your problem to localize the issue in the code. Also, such issues are not very common because the value obtained from an UB operation is usually nonsense (shifting past bitwidth, out of bounds array element, etc) so a compiler switching things around is just garbage-in-garbage-out. It is of course a serious issue if a program is actually depending on such a value for a crucial operation. That's how you get exploits, with or without the compiler doing something clever.

anonymoushn 5y ago
One of my least favorite bugs came from a loop that iterated from *p to *(p+n) (exclusive). For n=0 and values of p other than null, the loop body never executes. For n=0 and p=null, the loop body is allowed to execute, because null+0 is allowed to be any value.
- fooker 5y ago
  Nice example! Seems solvable by printf debugging though..
  - taneq 5y ago
    The fact that it’s fixable doesn’t mean it’s fun…
barrkel 5y ago
Miscompilations due to UB are normally silent and can remove checks written into the code for security purposes; e.g. checks for overflowing signed integers, or for null pointers. They're not harder to debug. Debugging them isn't the issue. The problem is knowing that the code in the editor doesn't correspond to the code under execution.
vlovich123 5y ago
All I can do is point out that very big names in CS (including Linus) disagree with you: http://www.yodaiken.com/wp-content/uploads/2018/05/ub-1.pdf http://www.yodaiken.com/wp-content/uploads/2018/05/ub-1.pdf This is probably because they’ve seen the effects of the foot gun first hand. They also recall what C/C++ looked like before the standards bodies made exploiting UB in compilers open season. There’s also a reason why Rust doesn’t have any UB in its safe dialect even though it sits on infrastructure capable of most/all the same possible optimizations via LLVM. There was in fact a direct kernel security issue UB exploitation caused in the Linux kernel whereas without that optimization there is no security issue. if I recall correctly the code looked something like: Some_value = ptr->value If (!ptr) { return; } This worked correctly in all cases before the compiler added the optimization. It worked less well after because the !ptr check was UB since it followed a dereference. It also wasn’t immediately obvious this code was broken because there was no diagnostic nor any indication that upgrading a compiler would suddenly elide the check. The value-add of such optimizations at scale vs the correctness issues exploiting it causes is questionable. The problem wasn’t that it’s not fixable. It’s that it takes time to find and you may not even find it until after it’s being exploited.
- UncleMeat 5y ago
  That code is buggy if ptr can be null. There isn't an especially rigorous distinction between a bug and a vuln. So saying "this was definitely a safe bug and the compiler elevated it to a vuln" can be a thing in very specific situations, but it also isn't generalizable in any way. Linus is an important figure, but there are plenty of world experts on the other side of this argument. And there is even a third side that states that not only should the compiler avoid optimizations based on UB assumptions, C implementations should instead function more like a virtual machine for a PDP-11 and have everything under the sun be defined (at great cost of performance). If you take an especially pessimistic view from the compiler, then really really basic stuff is almost impossible. A write to a dereferenced pointer is almost equivalent to a full program havoc. This invalidates almost all facts that the compiler knows and prevents virtually every optimization near that write. Heck, technically this can interfere with things like vtables and invalidate any sort of devirtualization, which is hugely valuable for performance.