2 ms·

My experience is that self-described "very experienced C system programmers" are simultaneously the people who are most in need of a good explainer on undefined

My experience is that self-described "very experienced C system programmers" are simultaneously the people who are most in need of a good explainer on undefined behavior and the most likely ones to throw a conniption fit halfway through and stop reading, for the hallmark of a good explainer on UB is that it will explain that a) it exists for a reason; b) no, just doing a "little" UB isn't safe; and c) it's not the compiler's fault that things go awry when you do UB, it's the programmer's fault. One of the blog posts I've long queued up for writing is "In defense of undefined behavior." It's only half-written, though, but the gist is justifying UB by pointing out that you can't optimize C code with it (via an example using pointer provenance), then pointing out why uninitialized values look weirder than you think by reference to the effects of system libraries, and then I would actually walk through why specification authors should reach for undefined behavior in various places.

raphlinus 2y ago
Oh hey, I also have "in defense of undefined behavior" in the queue of blog posts I'd like to write some time, with that exact title. What a coincidence. That said, it's unlikely to get written as I have things that are more specific to my actual research ahead of it. One of the things I'd want to say is that UB is a useful and accurate way to model what happens when, say, a program writes over memory used by the allocator. Languages like Odin might try to pretend they don't have UB, but in my opinion it's impossible to get there just by disabling certain compiler optimizations (see https://news.ycombinator.com/item?id=32800814 https://news.ycombinator.com/item?id=32800814 for an argument about this). I see UB as essentially a proof obligation, to be discharged in some other way. A really good way is to have UB in the intermediate representation, and compile a safe language into it (with unsafe escape hatches when needed). But there are other ways, including formal methods, rigorous testing, or just being a really smart solo programmer who's learned how to avoid UB and doesn't have to work in a team. Feel free to send me your draft.
- vlovich123 2y ago
  Raph, I think you may be using a different definition of UB than what compiler authors are using? As I understand it in the language sense of the word, UB technically allows the compiler to interpret the code however it wants. To me utility in UB are relying on some kind of well-defined behavior to result which would imply that you are either just relying on today’s behavior OR you are doing something that’s non-deterministic but not violating language rules? Or some intermediate definition where it’s both violating language rules but no future version of the compiler is likely to be able to detect the UB and change behavior? UB is very useful for compiler authors because they can apply very useful optimizations with “illegal” code and then emit illegal code constructs when they want those optimizations to apply. I have a hard time understanding how that’s useful to language users though.
  - raphlinus 2y ago
    The argument I have in mind is subtle and nuanced, and I didn't write clearly in that comment (the bit about the smart solo programmer was mostly sarcasm but with a grain of truth). But to try to answer: The value of UB is to clearly document what the obligations are for valid programs. It's not valuable to indiscriminately expose that to programmers at scale without some mechanism to discharge those obligations. I don't think C's choices for UB are defensible in a modern world, and for part of that evidence see how many misconceptions there are in this thread (just to pick one, that at least some people think the move to two's complement means that signed overflow is no longer UB). On the other hand, unsafe Rust's choice to include more UB is defensible (aliasing a mutable reference is UB in unsafe Rust but not UB in C) is defensible, as it makes the whole system safer. And Odin's approach (claiming there's no UB when there actually is UB) is even worse from a "clear communication" perspective. But maybe I should actually write the blog post some time.
    - vlovich123 2y ago
      Ah gotcha. Yeah it’s useful from a language perspective. One thing I wish unsafe Rust would do, given that there’s more potential for UB in unsafe Rust than in C/C++, would be some way to enforce more compiler checks against UB you didn’t intend to stumble across but I don’t know how that would work or if it’s even possible. But Rust unsafe is decidedly sharper than C++.
- pjmlp 2y ago
  Using protective gear, or making cars safer for crashes, also slows down physics versus not using them at all, yet lifes are saved every year where people would otherwise die or be crippled. As someone that rather prefers Wirth culture on programming languages, UB at the expense of safety isn't a clear win, that is why we end up with security exploits or hardware mitigations for UB based optmizations gone too far.
fsckboy 2y ago
>most likely ones to throw a conniption fit halfway through and stop reading listen to you, you didn't read the article it's clear, because the article doesn't agree with the rest of what you said. So, since you're not defending an article, you're just attacking me, which is pretty schmucky
- jcranmer 2y ago
  I did read the article. I have my disagreements with John Regehr on UB, though, as he's definitely in the camp of "let's try to specify it out entirely from the compiler," which doesn't entirely work.

raphlinus 2y ago

Oh hey, I also have "in defense of undefined behavior" in the queue of blog posts I'd like to write some time, with that exact title. What a coincidence. That said, it's unlikely to get written as I have things that are more specific to my actual research ahead of it. One of the things I'd want to say is that UB is a useful and accurate way to model what happens when, say, a program writes over memory used by the allocator. Languages like Odin might try to pretend they don't have UB, but in my opinion it's impossible to get there just by disabling certain compiler optimizations (see https://news.ycombinator.com/item?id=32800814 https://news.ycombinator.com/item?id=32800814 for an argument about this). I see UB as essentially a proof obligation, to be discharged in some other way. A really good way is to have UB in the intermediate representation, and compile a safe language into it (with unsafe escape hatches when needed). But there are other ways, including formal methods, rigorous testing, or just being a really smart solo programmer who's learned how to avoid UB and doesn't have to work in a team. Feel free to send me your draft.

vlovich123 2y ago
Raph, I think you may be using a different definition of UB than what compiler authors are using? As I understand it in the language sense of the word, UB technically allows the compiler to interpret the code however it wants. To me utility in UB are relying on some kind of well-defined behavior to result which would imply that you are either just relying on today’s behavior OR you are doing something that’s non-deterministic but not violating language rules? Or some intermediate definition where it’s both violating language rules but no future version of the compiler is likely to be able to detect the UB and change behavior? UB is very useful for compiler authors because they can apply very useful optimizations with “illegal” code and then emit illegal code constructs when they want those optimizations to apply. I have a hard time understanding how that’s useful to language users though.
- raphlinus 2y ago
  The argument I have in mind is subtle and nuanced, and I didn't write clearly in that comment (the bit about the smart solo programmer was mostly sarcasm but with a grain of truth). But to try to answer: The value of UB is to clearly document what the obligations are for valid programs. It's not valuable to indiscriminately expose that to programmers at scale without some mechanism to discharge those obligations. I don't think C's choices for UB are defensible in a modern world, and for part of that evidence see how many misconceptions there are in this thread (just to pick one, that at least some people think the move to two's complement means that signed overflow is no longer UB). On the other hand, unsafe Rust's choice to include more UB is defensible (aliasing a mutable reference is UB in unsafe Rust but not UB in C) is defensible, as it makes the whole system safer. And Odin's approach (claiming there's no UB when there actually is UB) is even worse from a "clear communication" perspective. But maybe I should actually write the blog post some time.
  - vlovich123 2y ago
    Ah gotcha. Yeah it’s useful from a language perspective. One thing I wish unsafe Rust would do, given that there’s more potential for UB in unsafe Rust than in C/C++, would be some way to enforce more compiler checks against UB you didn’t intend to stumble across but I don’t know how that would work or if it’s even possible. But Rust unsafe is decidedly sharper than C++.
pjmlp 2y ago
Using protective gear, or making cars safer for crashes, also slows down physics versus not using them at all, yet lifes are saved every year where people would otherwise die or be crippled. As someone that rather prefers Wirth culture on programming languages, UB at the expense of safety isn't a clear win, that is why we end up with security exploits or hardware mitigations for UB based optmizations gone too far.

fsckboy 2y ago

>most likely ones to throw a conniption fit halfway through and stop reading listen to you, you didn't read the article it's clear, because the article doesn't agree with the rest of what you said. So, since you're not defending an article, you're just attacking me, which is pretty schmucky

jcranmer 2y ago
I did read the article. I have my disagreements with John Regehr on UB, though, as he's definitely in the camp of "let's try to specify it out entirely from the compiler," which doesn't entirely work.