14 ms·

> compiler writers refuse to take responsibility for the bugs they introduced, even though the compiled code worked fine before the "optimizations". The excuse

• nabla9 2y ago
  Optimizing compilers that don't allow disabling all optimizations makes it impossible to write secure code with them. Must do it with assembly.
  • exe34 2y ago
    clang::optnone
    • nabla9 2y ago
      "Optimizing compilers that don't allow disabling __all__ optimizations"
      • exe34 2y ago
        do these exist? who's using them?
      • layer8 2y ago
        It’s not well-defined what counts as an optimization. For example, should every single source-level read access of a memory location go through all cache levels down to main memory, instead of, for example, caching values in registers? That would be awfully slow. But that question is one reason for UB.
        • kevingadd 2y ago
          Or writing code that relies on inlining and/or tail call optimization to successfully run at all without running out of stack... We've got some code that doesn't run if compiled O0 due to that.
  • cmeacham98 2y ago
    If your "secure" code is not secure because of a compiler optimization it is fundamentally incorrect and broken.
    • nabla9 2y ago
      It's secure code we use. I'm sure you know who DJB is.
      • jjuhl 2y ago
        Why is knowing who the author is relevant? Either what he posts is correct or it is not, who the person is is irrelevant.
    • hedgehog 2y ago
      There is a fundamental difference of priorities between the two worlds. For most general application code any optimization is fine as long as the output is correct. In security critical code information leakage from execution time and resource usage on the chip matters but that essentially means you need to get away from data-dependent memory access patterns and flow control.
      • account42 2y ago
        Then such code needs to be written in a language that actually makes the relevant timing guarantees. That language may be C with appropriate extensions but it certainly is not C with whining that compilers don't apply my special requirements to all code.
        • hedgehog 2y ago
          That argument would make more sense if such a language was widely available but today in practice it isn't so we live in the universe of less ideal solutions. Actually it doesn't really respond to DJB's point anyway, his case here is that the downstream labor cost of compiler churn exceeds the actual return in performance gains from new features and that a change in policy could give security-related code a more predictable target without requiring a whole new language or toolchain. For what it's worth I think the better solution will end up being something like constant-time function annotations (not stopping new compiler features) but I don't discount his view that absent human nature maybe we would be better of focusing compiler dev on correctness and stability.
  • bluGill 2y ago
    If you have ub then you have a bug and there is some system that will show it. It isn't hard to write code without ub.
    • bigstrat2003 2y ago
      It is, in fact, pretty hard as evidenced by how often programmers fail at it. The macho attitude of "it's not hard, just write good code" is divorced from observable reality.
      • bluGill 2y ago
        People write buffer overflows because and memory leaks they are not coreful. The rest of ub are things I have never seen despite running sanitizers and a large codebase.
        • saagarjha 2y ago
          Perhaps you’re not looking all that hard.
          • bluGill 2y ago
            Sanitizers are very good at finding ub.
            • deleted 2y ago
              [deleted]
            • saagarjha 2y ago
              Sure. That's just a function of how much UB there is, rather than them catching it all.
      • tbrownaw 2y ago
        Staying under the speed limit is, in fact, pretty hard as evidenced by how often drivers fail at it.
  • Rusky 2y ago
    Disabling all optimizations isn't even enough- fundamentally what you need is a much narrower specification for how the source language maps to its output. Even -O0 doesn't give you that, and in fact will often be counterproductive (e.g. you'll get branches in places that the optimizer would have removed them). The problem with this is that no general purpose compiler wants to tie its own hands behind its back in this way, for the benefit of one narrow use case. It's not just that it would cost performance for everyone else, but also that it requires a totally different approach to specification and backwards compatibility, not to mention deep changes to compiler architecture. You almost may as well just design a new language, at that point.
    • amluto 2y ago
      > You almost may as well just design a new language, at that point. Forget “almost”. Go compile this C code: void foo(int *ptr) { free(ptr); *ptr = 42; } This is UB. And it has nothing whatsoever to do with optimizations — any sensible translation to machine code is a use-after-free, and an attacker can probably find a way to exploit that machine code to run arbitrary code and format your disk. If you don’t like this, use a language without UB. But djb wants something different, I think: a way to tell the compiler not to introduce timing dependencies on certain values. This is a nice idea, but it needs hardware support! Your CPU may well implement ALU instructions with data-dependent timing. Intel, for example, reserves the right to do this unless you set an MSR to tell it not to. And you cannot set that MSR from user code, so what exactly is a compiler supposed to do? https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/best-practices/data-operand-independent-timing-isa-guidance.html https://www.intel.com/content/www/us/en/developer/articles/t...
      • Rusky 2y ago
        I am not talking about UB at all. I am talking about the same constant-time stuff that djb's post is talking about.
        • SAI_Peregrinus 2y ago
• thayne 2y ago
  The problem is that c and c++ have a ridiculous amount of undefined behavior, and it is extremely difficult to avoid all of it. One of the advantages of rust is it confines any potential UB to unsafe blocks. But even in rust, which has defined behavior in a lot of places that are UB in c, if you venture into unsafe code, it is remarkable easy to accidentally run into subtle UB issues.
  • layer8 2y ago
    It’s true that UB is not intuitive at first, but “ridiculous amount” and “difficult to avoid” is overstating it. You have to have a proof-writing mindset when coding, but you do get sensitized to the pitfalls once you read up on what the language constructs actually guarantee (and don’t guarantee), and it’s not that much more difficult than, say, avoiding panics in Rust.
    • pjmlp 2y ago
      So surely you know by hear the circa 200 use cases documented in ISO C, and the even greater list documented in ISO C++ standard documents. Because, me despite knowing both since the 1990's, I rather leave that to static analysis tools.
    • tomjakubowski 2y ago
      In my experience it is very easy to accidentally introduce iterator invalidation: it starts with calling a callback while iterating, add some layers of indirection, and eventually somebody will add some innocent looking code deep down the call stack which ends up mutating the collection while it's being iterated.
      • layer8 2y ago
        I can tell you that this happens in Java as well, which doesn’t have undefined behavior. That’s just the nature of mutable state in combination with algorithms that only work while the state remains unmodified.
        • 2y ago
• js2 2y ago
  I think you're replying to a strawman. Here's the full quote: > The excuse for not taking responsibility is that there are "language standards" saying that these bugs should be blamed on millions of programmers writing code that bumps into "undefined behavior", rather than being blamed on the much smaller group of compiler writers subsequently changing how this code behaves. These "language standards" are written by the compiler writers. > Evidently the compiler writers find it more important to continue developing "optimizations" than to have computer systems functioning as expected. Developing "optimizations" seems to be a very large part of what compiler writers are paid to do. The argument is that the compiler writers are themselves the ones deciding what is and isn't undefined, and they are defining those standards in such a way as to allow themselves latitude for further optimizations. Those optimizations then break previously working code. The compiler writers could instead choose to prioritize backwards compatibility, but they don't. Further, these optimizations don't meaningfully improve the performance of real world code, so the trade-off of breaking code isn't even worth it. That's the argument you need to rebut.
  • quohort 2y ago
    Perhaps the solution is also to reign in the language standard to support stricter use cases. For example, what if there was a constant-time { ... }; block in the same way you have extern "C" { ... }; . Not only would it allow you to have optimizations outside of the block, it would also force the compiler to ensure that a given block of code is always constant-time (as a security check done by the compiler).
    • munificent 2y ago
      > Perhaps the solution is also to reign in the language standard to support stricter use cases. Here's a nine-year-old comment from the author asking for exactly that: https://groups.google.com/g/boring-crypto/c/48qa1kWignU/m/o8GGp2K1DAAJ https://groups.google.com/g/boring-crypto/c/48qa1kWignU/m/o8...
      • fuhsnn 2y ago
        That thread already spawned a GCC-wiki page https://gcc.gnu.org/wiki/boringcc https://gcc.gnu.org/wiki/boringcc, with a quote in bold: >The only thing stopping gcc from becoming the desired boringcc is to find the people willing to do the work. And frankly, nine years is enough time to build a C compiler from scratch.
• duped 2y ago
  I think this is actually a mistake by the author since the rant is mostly focused on implementation defined behavior, not undefined. The examples they give are all perfectly valid code. The specific bugs they're talking about seem to be compiler optimizations that replace bit twiddling arithmetic into branches, which isn't a safe optimization if the bit twiddling happens in a cryptographic context because it opens the door for timing attacks. I don't think it's correct to call either the source code or compiler buggy, it's the C standard that is under specified to the author's liking and it creates security bugs on some targets. Ultimately though I can agree with the C standard authors that they cannot define the behavior of hardware, they can only define the semantics for the language itself. Crypto guys will have to suffer because the blame is on the hardware for these bugs, not the software.
  • gabrielhidasy 2y ago
    Where suffer means "not be lazy, implement the assembly for your primitives in a lib, optimize it as best as you can without compromising security, do not let the compiler 'improve' it"
    • Dylan16807 2y ago
      But then you're not writing C, except maybe as some wrappers. Wanting to use C isn't laziness. Making it nearly unfeasible to use C is the most suffering a C compiler can inflict.
      • g15jv2dp 2y ago
        There's no reason that C should be suitable for every purpose under the sun.
        • Dylan16807 2y ago
          Fiddling some bits cross-platform is supposed to be one of them.
• moomin 2y ago
  There are only two models of UB that are useful to compiler users: 1) This is a bad idea and refuse to compile. 2) Do something sensible and stable. Silently fail and generate impossible to predict code is a third model that is only of use to compiler writers. Hiding behind the spec benefits no actual user.
  • opnitro 2y ago
    I think this is a point of view that seems sensible, but probably hasn't really thought through how this works. For example some_array[i] What should the compiler emit here? Should it emit a bounds check? In the event the bounds check fails, what should it do? It is only through the practice of undefined behavior that the compiler can consistently generate code that avoids the bounds check. (We don't need it, because if `i` is out-of-bounds then it's undefined behavior and illegal). If you think this is bad, then you're arguing against memory unsafe languages in general. A sane position is the one the Rust takes, which is by default, yes indeed you should always generate the bounds check (unless you can prove it always succeeds). But there will likely always be hot inner loops where we need to discharge the bounds checks statically. Ideally that would be done with some kind of formal reasoning support, but the industry is far that atm. For a more in depth read: https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213
    • Dylan16807 2y ago
      > What should the compiler emit here? It should emit an instruction to access memory location some_array + i. That's all most people that complain about optimizations on undefined behavior want. Sometimes there are questions that are hard to answer, but in a situation like this, the answer is "Try it and hope it doesn't corrupt memory." The behavior that's not wanted is for the compiler to wildly change behavior on purpose when something is undefined. For example, the compiler could optimize if(foo) { misbehaving_code(); return puppies; } else { delete_data(); } into delete_data();
      • g15jv2dp 2y ago
• userbinator 2y ago
  Calling the compiler buggy for not doing what you want when you commit Undefined Behavior is like calling dd buggy for destroying your data when you call it with the wrong arguments. No, it's like calling dd buggy for deliberately zeroing all your drives when you call it with no arguments. How did we let pedantic brainless "but muh holy standards!!!1" religious brigading triumph over common sense? The standards left things undefined in the hopes that the language would be more widely applicable and implementers would give those areas thought themselves and decide the right thing. Not so that compiler writers can become adversarial smartasses. It even suggests that "behaving in a manner characteristic of the environment" is a possible outcome of UB, which is what "the spirit of C" is all about. In my observations this gross exploitation of UB started with the FOSS compilers, GCC and Clang being the notable examples. MSVC or ICC didn't need to be so crazy, and yet they were very competitive, so I don't believe claims that UB is necessary for optimisation. The good thing about FOSS is that those in power can easily be changed. Perhaps it's time to fork, fix, and fight back.
  • jjnoakes 2y ago
    > The standards left things undefined in the hopes that the language would be more widely applicable and implementers would give those areas thought themselves and decide the right thing. That sounds like implementation-defined behavior, not undefined behavior.
    • userbinator 2y ago
      Same difference. You still have to think about what's right.
      • jjnoakes 2y ago
        They are different by definition.
  • bee_rider 2y ago
    > The good thing about FOSS is that those in power can easily be changed. Perhaps it's time to fork, fix, and fight back. Huzzah! Lead on, then.
• aleksi 2y ago
  Russ Cox has a nice article about it: C and C++ Prioritize Performance over Correctness (https://research.swtch.com/ub https://research.swtch.com/ub)
• GTP 2y ago
  We can debate whether it's reasonable or not to optimize code based on undefined behavior. But we should at least have the compiler emit a warning when this happens. Just like we have the notorious "x makes an integer from a pointer without a cast", we could have warnings for when the compiler decides to not emit the code for an if branch checking for a null pointer or an instruction zeroing some memory right before deallocation (I think this is not UB, but still a source of security issues due to extreme optimizations).
• matheusmoreira 2y ago
  Plenty of undefined behavior is actually perfectly good code the compiler has no business screwing up in any way whatsoever. This is C, we do evil things like cast pointers to other types and overlay structures onto byte buffers. We don't really want to hear about "undefined" nonsense, we want the compiler to accept the input and generate the code we expect it to. If it's undefined, then define it. This attitude turns bugs into security vulnerabilities. There's a reason the Linux kernel is compiled with -fwrapv -fno-strict-aliasing -fno-delete-null-pointer-checks and probably many more sanity restoring flags. Those flags should actually be the default for every C project.
• nine_k 2y ago
  I would say that allowing undefined behavior is a bug in itself. It was an understandable mistake for 1970, especially for such a hacker language as C. But now if a compiler can detect UD, it should warn you about it (and mostly it does by default), and you should treat that warning as an error. So, well, yes, if the bug is due to triggering UD, some blame should fall on the developer, too.