4 ms·

You should read the post you are replying to again, as it is well written and precise. Basically, the intent of the proposal is understandable but it does not

by obl 8y ago

You should read the post you are replying to again, as it is well written and precise. Basically, the intent of the proposal is understandable but it does not make sense as written. The spec does not explain how to compile C code, it gives you a set of rules so that you could in theory take some piece of C source and a full execution trace and verify whether that trace was a conforming execution for that C code. You take the example of the negative shift again, and it is addressed in the post you respond to : it's a perfectly fine proposal to demote negative shifts from UB to something less permissive to the compiler. However, if you do so, of course, you have to frame what behaviors are accepted for a program doing such a negative shifts, just as you did in your post ! So let's say : arbitrary (but consistent) values are fine, termination is fine. That's great because it means we can still compile a shift to a CPU shift and there's no UB anymore. Ok. Same goes for signed overflow. However this is a case by case process, and you will absolutely not be able to ascribe meaning this way to many forms of UB. For example, as the GP said, storing through an invalid pointer. If you want to compile a pointer store to a store instruction without runtime checks, then possible observable behavior include : other variable changing values, function call not returning to their calling context, and even of course running a full ROP chain exploit ! It's no surprise that in that case the specs says that the set of allowed observable behaviors is anything.

jstimpfle 8y ago
> It's no surprise that in that case the specs says that the set of allowed observable behaviors is anything. And yet UB is frequently described as this scary monster that will eat your hard drive. Sure there are more UBs in C than there would need to be if the standard restricted more on contemporary hardware. I think it still allows ones' complement representation for example? But I guess it shouldn't be so bad as long as compilers do not exploit the freedom to do anything that comes from UB, more than they need to on a particular architecture?
- cesarb 8y ago
  Compilers do not "exploit the freedom to do anything that comes from UB", they just assume that the program does not have UB. This might or might not lead the compiler to behave as if it was doing anything it wants, but it's not screwing the programmer on purpose. As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should.
  - jstimpfle 8y ago
    Let me take a shift operation as an example. On architectures where it's no problem to "overshift" (e.g. the operation always results in 0), the compiler could be just fine and leave shifts alone. Maybe a 0 is exactly what the user wanted! Or, the compiler could assume that the programmer made sure that overshifting never happens, since it's UB. Based on that assumption, it could make more aggressive (and possibly totally insignificant) optimizations by inferring other assumptions that don't align with the programmer's idea anymore. I never encountered such a sitation myself, but there are stories like that on the internet, where for example the Linux kernel was broken. Another scenario, the user might always do an operation that sometimes results in a random value (which is UB), but in the end only use the resulting value if it was not UB. The compiler instead could assume the operation can never happen in the first place. Again, there is a misalignment here in how the compiler reasoned and how the programmer reasoned (closer to the actual hardware). So, the difficult problem is probably defining the UB configurations as narrowly as possible. It might be a good idea to specify UB per architecture in some cases, but then it would lead to a lot of complexity. In any case a guideline for compilers should be to minimize surprises, which means not applying many optimizations around UB. That's easier said than done I guess.
  - jstimpfle 8y ago
    > As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should. I'd figure it's much more likely to happen as a result of a logic error introduced by the programmer, without any UB involved.
    - cesarb 8y ago
      There's also this infamous example of UB: https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html https://kristerw.blogspot.com/2017/09/why-undefined-behavior... (explanation: https://blog.tchatzigiannakis.com/undefined-behavior-can-literally-erase-your-hard-disk/ https://blog.tchatzigiannakis.com/undefined-behavior-can-lit...)
vyodaiken 8y ago
Argument like yours seem to rely on an incorrect understanding of the design of C. Languages like Java encapsulate program execution in a runtime that, in theory, can catch and manage all sorts of programmer errors that C does not catch. That's ok. If the programmer references past the end of an array, the compiler may catch and object, it may simply compile the instructions and let the program step on its own data structures or encounter a memory fault or even a protection violation if the architecture provides some type protection. And it's bizarre to see this warning about ROP exploit, when current "optimization" based on UB is a source of ROP exploit vulnerability. One cannot reliably put protections in code against bad pointers if upstream errors are grounds for deleting the guards.

by obl 8y ago

jstimpfle 8y ago

> It's no surprise that in that case the specs says that the set of allowed observable behaviors is anything. And yet UB is frequently described as this scary monster that will eat your hard drive. Sure there are more UBs in C than there would need to be if the standard restricted more on contemporary hardware. I think it still allows ones' complement representation for example? But I guess it shouldn't be so bad as long as compilers do not exploit the freedom to do anything that comes from UB, more than they need to on a particular architecture?

cesarb 8y ago
Compilers do not "exploit the freedom to do anything that comes from UB", they just assume that the program does not have UB. This might or might not lead the compiler to behave as if it was doing anything it wants, but it's not screwing the programmer on purpose. As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should.
- jstimpfle 8y ago
  Let me take a shift operation as an example. On architectures where it's no problem to "overshift" (e.g. the operation always results in 0), the compiler could be just fine and leave shifts alone. Maybe a 0 is exactly what the user wanted! Or, the compiler could assume that the programmer made sure that overshifting never happens, since it's UB. Based on that assumption, it could make more aggressive (and possibly totally insignificant) optimizations by inferring other assumptions that don't align with the programmer's idea anymore. I never encountered such a sitation myself, but there are stories like that on the internet, where for example the Linux kernel was broken. Another scenario, the user might always do an operation that sometimes results in a random value (which is UB), but in the end only use the resulting value if it was not UB. The compiler instead could assume the operation can never happen in the first place. Again, there is a misalignment here in how the compiler reasoned and how the programmer reasoned (closer to the actual hardware). So, the difficult problem is probably defining the UB configurations as narrowly as possible. It might be a good idea to specify UB per architecture in some cases, but then it would lead to a lot of complexity. In any case a guideline for compilers should be to minimize surprises, which means not applying many optimizations around UB. That's easier said than done I guess.
- jstimpfle 8y ago
  > As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should. I'd figure it's much more likely to happen as a result of a logic error introduced by the programmer, without any UB involved.
  - cesarb 8y ago
    There's also this infamous example of UB: https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html https://kristerw.blogspot.com/2017/09/why-undefined-behavior... (explanation: https://blog.tchatzigiannakis.com/undefined-behavior-can-literally-erase-your-hard-disk/ https://blog.tchatzigiannakis.com/undefined-behavior-can-lit...)

vyodaiken 8y ago

Argument like yours seem to rely on an incorrect understanding of the design of C. Languages like Java encapsulate program execution in a runtime that, in theory, can catch and manage all sorts of programmer errors that C does not catch. That's ok. If the programmer references past the end of an array, the compiler may catch and object, it may simply compile the instructions and let the program step on its own data structures or encounter a memory fault or even a protection violation if the architecture provides some type protection. And it's bizarre to see this warning about ROP exploit, when current "optimization" based on UB is a source of ROP exploit vulnerability. One cannot reliably put protections in code against bad pointers if upstream errors are grounds for deleting the guards.