5 ms·

Undefined behavior is not the problem that people make it to be. First of all, undefined behavior is well understood by compilers, in fact it is exactly exploit

Undefined behavior is not the problem that people make it to be. First of all, undefined behavior is well understood by compilers, in fact it is exactly exploited by compilers to make code run faster. The only thing you need to solve UB is to ask compilers to stop exploiting it (which normally can be done by reducing optimization). And of course you can rewrite your code to stop relying on UB. Despite all the complains, I have never seen code suffering from UB that couldn't be fixed.

thesuperbigfrog 6y ago
>> undefined behavior is well understood by compilers The security implications of undefined behavior are poorly understood by software developers: https://arc.aiaa.org/doi/pdf/10.2514/1.I010699 https://arc.aiaa.org/doi/pdf/10.2514/1.I010699
MaxBarraclough 6y ago
> Undefined behavior is not the problem that people make it to be No, undefined behaviour is a serious problem, especially regarding security. There's a long history of serious security problems in C and C++ codebases due to unintended invocation of undefined behaviour. These issues continue to arise even in well-resourced C/C++ projects with highly skilled developers, such as the Linux kernel and Chromium.
- jfoutz 6y ago
  It’s pretty sweet for optimization though. Doing Nothing is almost always faster than the alternatives.
  - MaxBarraclough 6y ago
    The reasons for undefined behaviour being in the C/C++ language standards are performance and the ability to naturally support all sorts of different platforms, but I'm not sure the performance argument really holds up with modern optimising compilers. I admit I don't have hard numbers to hand, and sadly it's rather rare to see decent rigorous comparisons, but I don't think C and C++ have that much of a performance advantage over, say, Ada or Safe Rust. My inner pedant feels it's necessary to note that Ada is still an unsafe language, but still, it's much less unsafe than C. It also has excellent support for optionally disabling all sorts of runtime safety checks, whereas the design of C and C++ make it extremely difficult to implement such checks as 'opt-in' features in a compiler.
    - astrange 6y ago
      It actually is important, but it's mostly important when your program gets deployed across architectures. If you're only writing for one architecture (say x86) then you can write the C to just work well there. One example is that undefined signed overflow is important when the compiler might need to rearrange a loop index - PPC prefers to count down, x86 doesn't care. Anyway, I think undefined behavior + ubsan is better than defining all behavior. If something's undefined you know it's a bug every time you see it. If it's defined, how do you know it's wrong?
      - MaxBarraclough 6y ago
        > If you're only writing for one architecture (say x86) then you can write the C to just work well there. > One example is that undefined signed overflow is important when the compiler might need to rearrange a loop index - PPC prefers to count down, x86 doesn't care. Signed overflow is undefined behaviour regardless of the target hardware architecture. The compiler is permitted to assume the absence of signed overflow and to optimise accordingly. Unless the compiler documentation specifically says its ok, you still have an undefined behaviour problem. It might happen to work fine, sure, but if you're serious about writing correct programs you should be aiming to deliver a program which is correct-by-definition rather than correct-by-coincidence. > I think undefined behavior + ubsan is better than defining all behavior It isn't. That's why Rust makes such a big deal of its Safe Rust subset. It's also a major selling point of SPARK Ada for safety-critical software. It's tremendously valuable to be able to categorically close the door on a whole family of potentially serious and difficult to detect bugs. > If something's undefined you know it's a bug every time you see it. No, you absolutely don't. I already mentioned that high-profile projects like Chromium and the Linux kernel continue to face security vulnerabilities arising from unintended invocation of undefined behaviour. Section 7 of the paper discusses undefined behaviour but doesn't really explore its full consequences, so instead I suggest reading [0] and [1]. Undefined behaviour means exactly that: if undefined behaviour has been invoked at runtime, the behaviour of the program is not constrained by the C/C++ standard. The program is not required to explode loudly, it can do anything. It isn't required to behave the same way each time. Hopefully it will explode loudly, but it's possible everything will seem to be fine. In the worst case the undefined behaviour leads to a serious safety issue or security vulnerability. Undefined behaviour is even permitted to 'time travel'. [0][2] > If it's defined, how do you know it's wrong? You use exceptions or some other well-defined means of detecting and handling runtime errors. For example, Java's NullPointerException and Ada's Constraint_Error. [0] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [1] https://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html https://blog.llvm.org/2011/05/what-every-c-programmer-should... [2] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63...
- coliveira 6y ago
  Undefined behavior is a problem for projects that are not testing certain conditions under certain compiler configurations. Every time you see these problems happening, the reason is that developers not only wrote incorrect (undefined) code, but also didn't test the code path that was affected. While I agree that a compiler should do a better job, ultimately the responsibility is on programmers (especially with the C language, where this has always been the unwritten contract).
  - MaxBarraclough 6y ago
    > Undefined behavior is a problem for projects that are not testing certain conditions under certain compiler configurations. Like I said, undefined behaviour is a problem even for Chromium and the Linux kernel. We've seen that just be diligent isn't a solution. We're way past that now. > Every time you see these problems happening, the reason is that developers not only wrote incorrect (undefined) code, but also didn't test the code path that was affected. Technically true, but not insightful. Testing can never be exhaustive, so this observation doesn't light the way to a solution. > While I agree that a compiler should do a better job, ultimately the responsibility is on programmers This statement is true of the C language, but it isn't a solution, it's the problem-statement. It's not true of a language like Safe Rust, where the language itself closes the door on undefined behaviour, and where it is not the programmer's responsibility to avoid invoking undefined behaviour.
    - coliveira 6y ago
      It is well known that undefined behavior are not the only bugs happening in codebases like Chromium and Linux. So pointing to these projects doesn't magically make the bug different from other categories of bugs that plague all kinds of projects, even in other languages. The major compilers already have ways to test for undefined behavior such as -fsanitize=undefined. Projects need to use these flags and test more.
      - MaxBarraclough 6y ago
        > pointing to them doesn't magically make the bug different from other categories of bugs that plague all kinds of projects, even in other languages Undefined behaviour is responsible for a non-trivial fraction of the security vulnerabilities of C and C++ codebases. Undefined behaviour in an application can be entirely eliminated by writing the application in a safe language. That's the point. > The major compilers already have ways to test for undefined behavior such as -fsanitize=undefined. Projects need to use these flags and test more. Do you really think the Chromium team isn't aware of that flag in GCC and Clang? If there were an easy fix to the problem of accidental invocation of undefined behaviour, the problem would have gone away years ago. It's useful for a C/C++ compiler to offer to add runtime checks for a subset of the possible causes of undefined behaviour. We agree more projects should use such tools. As we're seeing, though, this isn't a silver bullet. Even extremely well-resourced and security-sensitive codebases end up with UB issues.
citrin_ru 6y ago
1. Fixing/avoiding UBs requires discipline (and time) not all programmers have 2. Many programmers pointed to UB in their code would argue that it is not a problem at all - especially if code works for them and users don't complain about bugs caused by UB. It would be interesting to make a following experiment: * find and patch UBs in multiple opensource projects which are relatively popular (at least used not only by an author) * send pull requests wich UB fixes and see which fraction will be closed as "won't fix". It requires a lot of time, but may show that many developers don't care about UB.
- MaxBarraclough 6y ago
  > 1. Fixing/avoiding UBs requires discipline (and time) not all programmers have 2. Many programmers pointed to UB in their code would argue that it is not a problem at all These points aren't wrong, but it understates how pernicious undefined behaviour can be. As I stated earlier, even well-resourced high-profile codebases like Chromium and the Linux kernel play cat-and-mouse with security issues originating in undefined behaviour. It's not just a matter of hiring diligent coders and using the usual quality-control processes (testing, code reviews, static analysis). It's possible for C/C++ code to be entirely free of undefined behaviour, but in practice this only seems to happen when formal methods are used, or when people use code-generators/transpilers to avoid writing C/C++ by hand.

thesuperbigfrog 6y ago

>> undefined behavior is well understood by compilers The security implications of undefined behavior are poorly understood by software developers: https://arc.aiaa.org/doi/pdf/10.2514/1.I010699 https://arc.aiaa.org/doi/pdf/10.2514/1.I010699

MaxBarraclough 6y ago

> Undefined behavior is not the problem that people make it to be No, undefined behaviour is a serious problem, especially regarding security. There's a long history of serious security problems in C and C++ codebases due to unintended invocation of undefined behaviour. These issues continue to arise even in well-resourced C/C++ projects with highly skilled developers, such as the Linux kernel and Chromium.

jfoutz 6y ago
It’s pretty sweet for optimization though. Doing Nothing is almost always faster than the alternatives.
- MaxBarraclough 6y ago
  The reasons for undefined behaviour being in the C/C++ language standards are performance and the ability to naturally support all sorts of different platforms, but I'm not sure the performance argument really holds up with modern optimising compilers. I admit I don't have hard numbers to hand, and sadly it's rather rare to see decent rigorous comparisons, but I don't think C and C++ have that much of a performance advantage over, say, Ada or Safe Rust. My inner pedant feels it's necessary to note that Ada is still an unsafe language, but still, it's much less unsafe than C. It also has excellent support for optionally disabling all sorts of runtime safety checks, whereas the design of C and C++ make it extremely difficult to implement such checks as 'opt-in' features in a compiler.
  - astrange 6y ago
    It actually is important, but it's mostly important when your program gets deployed across architectures. If you're only writing for one architecture (say x86) then you can write the C to just work well there. One example is that undefined signed overflow is important when the compiler might need to rearrange a loop index - PPC prefers to count down, x86 doesn't care. Anyway, I think undefined behavior + ubsan is better than defining all behavior. If something's undefined you know it's a bug every time you see it. If it's defined, how do you know it's wrong?
    - MaxBarraclough 6y ago
      > If you're only writing for one architecture (say x86) then you can write the C to just work well there. > One example is that undefined signed overflow is important when the compiler might need to rearrange a loop index - PPC prefers to count down, x86 doesn't care. Signed overflow is undefined behaviour regardless of the target hardware architecture. The compiler is permitted to assume the absence of signed overflow and to optimise accordingly. Unless the compiler documentation specifically says its ok, you still have an undefined behaviour problem. It might happen to work fine, sure, but if you're serious about writing correct programs you should be aiming to deliver a program which is correct-by-definition rather than correct-by-coincidence. > I think undefined behavior + ubsan is better than defining all behavior It isn't. That's why Rust makes such a big deal of its Safe Rust subset. It's also a major selling point of SPARK Ada for safety-critical software. It's tremendously valuable to be able to categorically close the door on a whole family of potentially serious and difficult to detect bugs. > If something's undefined you know it's a bug every time you see it. No, you absolutely don't. I already mentioned that high-profile projects like Chromium and the Linux kernel continue to face security vulnerabilities arising from unintended invocation of undefined behaviour. Section 7 of the paper discusses undefined behaviour but doesn't really explore its full consequences, so instead I suggest reading [0] and [1]. Undefined behaviour means exactly that: if undefined behaviour has been invoked at runtime, the behaviour of the program is not constrained by the C/C++ standard. The program is not required to explode loudly, it can do anything. It isn't required to behave the same way each time. Hopefully it will explode loudly, but it's possible everything will seem to be fine. In the worst case the undefined behaviour leads to a serious safety issue or security vulnerability. Undefined behaviour is even permitted to 'time travel'. [0][2] > If it's defined, how do you know it's wrong? You use exceptions or some other well-defined means of detecting and handling runtime errors. For example, Java's NullPointerException and Ada's Constraint_Error. [0] https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [1] https://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html https://blog.llvm.org/2011/05/what-every-c-programmer-should... [2] https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=633 https://devblogs.microsoft.com/oldnewthing/20140627-00/?p=63...
coliveira 6y ago
Undefined behavior is a problem for projects that are not testing certain conditions under certain compiler configurations. Every time you see these problems happening, the reason is that developers not only wrote incorrect (undefined) code, but also didn't test the code path that was affected. While I agree that a compiler should do a better job, ultimately the responsibility is on programmers (especially with the C language, where this has always been the unwritten contract).
- MaxBarraclough 6y ago
  > Undefined behavior is a problem for projects that are not testing certain conditions under certain compiler configurations. Like I said, undefined behaviour is a problem even for Chromium and the Linux kernel. We've seen that just be diligent isn't a solution. We're way past that now. > Every time you see these problems happening, the reason is that developers not only wrote incorrect (undefined) code, but also didn't test the code path that was affected. Technically true, but not insightful. Testing can never be exhaustive, so this observation doesn't light the way to a solution. > While I agree that a compiler should do a better job, ultimately the responsibility is on programmers This statement is true of the C language, but it isn't a solution, it's the problem-statement. It's not true of a language like Safe Rust, where the language itself closes the door on undefined behaviour, and where it is not the programmer's responsibility to avoid invoking undefined behaviour.
  - coliveira 6y ago
    It is well known that undefined behavior are not the only bugs happening in codebases like Chromium and Linux. So pointing to these projects doesn't magically make the bug different from other categories of bugs that plague all kinds of projects, even in other languages. The major compilers already have ways to test for undefined behavior such as -fsanitize=undefined. Projects need to use these flags and test more.
    - MaxBarraclough 6y ago
      > pointing to them doesn't magically make the bug different from other categories of bugs that plague all kinds of projects, even in other languages Undefined behaviour is responsible for a non-trivial fraction of the security vulnerabilities of C and C++ codebases. Undefined behaviour in an application can be entirely eliminated by writing the application in a safe language. That's the point. > The major compilers already have ways to test for undefined behavior such as -fsanitize=undefined. Projects need to use these flags and test more. Do you really think the Chromium team isn't aware of that flag in GCC and Clang? If there were an easy fix to the problem of accidental invocation of undefined behaviour, the problem would have gone away years ago. It's useful for a C/C++ compiler to offer to add runtime checks for a subset of the possible causes of undefined behaviour. We agree more projects should use such tools. As we're seeing, though, this isn't a silver bullet. Even extremely well-resourced and security-sensitive codebases end up with UB issues.

citrin_ru 6y ago

1. Fixing/avoiding UBs requires discipline (and time) not all programmers have 2. Many programmers pointed to UB in their code would argue that it is not a problem at all - especially if code works for them and users don't complain about bugs caused by UB. It would be interesting to make a following experiment: * find and patch UBs in multiple opensource projects which are relatively popular (at least used not only by an author) * send pull requests wich UB fixes and see which fraction will be closed as "won't fix". It requires a lot of time, but may show that many developers don't care about UB.

MaxBarraclough 6y ago
> 1. Fixing/avoiding UBs requires discipline (and time) not all programmers have 2. Many programmers pointed to UB in their code would argue that it is not a problem at all These points aren't wrong, but it understates how pernicious undefined behaviour can be. As I stated earlier, even well-resourced high-profile codebases like Chromium and the Linux kernel play cat-and-mouse with security issues originating in undefined behaviour. It's not just a matter of hiring diligent coders and using the usual quality-control processes (testing, code reviews, static analysis). It's possible for C/C++ code to be entirely free of undefined behaviour, but in practice this only seems to happen when formal methods are used, or when people use code-generators/transpilers to avoid writing C/C++ by hand.