29 ms·

Boringcc: Proposal for a C compiler for secure software

274 points by DCKing 11y ago

joshka 11y ago
Non mobile link: https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWignU/o8GGp2K1DAAJ https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWi...
atemerev 11y ago
Writing secure software in C is just next to impossible. Memory corruption, arbitrary pointers, deep magic. Even pure assembly is more secure, as everything is explicit and well-defined there. There are lots of safer languages than C, and performance is not everything.
- stcredzero 11y ago
  Is there a language that has powerful tools for dealing with timing attacks and can facilitate correctness proofs?
  - TheMagicHorsey 11y ago
    Ocaml
    - sbierwagen 11y ago
      Isn't ocaml garbage collected? Won't the GC pauses leak timing info?
      - bmm6o 11y ago
        If you are generating different amounts of garbage depending on your secrets, that's already showing up as leaked timing information.
  - nickpsecurity 11y ago
    You can deal with timing attacks using Kemmerer's Shared Resource Matrix to find those and other covert channels. Then, you just look at the bandwidth of them along with design alternatives that don't leak timing. You can sometimes make the timing predictable or non-data-driven at interface level to nullify that. An example comes from military-grade stuff way back that transmitted in fixed-length, fixed-rate packets. Markus Ottela added that to Tinfoil Chat at my recommendation. Far as correctness proofs, a combo of Ada 2012 with Design-by-Contract and SPARK is probably best at that. Far as app level, just program in a functional style with abstract state machines. Make it more concrete and detailed, replicating analysis or tests at each level. Eventually, you run the code checkers on the result and throw in some fuzz testing. No easy route, though, that I'm aware. Highly assured software and system design is a painstaking process. Avoiding risky stuff, defining all success/fail traces, failing safe, and static analysis to preempt bugs are the strongest route. Plus source-to-object correspondence to be sure compiler didn't eliminate safety checks or introduce problems. Spark Pro supports that.
- nine_k 11y ago
  I heard Rust can be pretty efficient? OTOH a heavily restricted C, closer to assembly than to Rust, could still be useful: it would be close to the metal but still preserve portability. Making such a language a strict subset of C would additionally help early porting efforts.
  - protomyth 11y ago
    Ada can be amazingly efficient. It just isn't cool. I've often wondered what would happen if someone pulled a reskining of the language (ala VB.NET >> C#) what would happen.
    - kevin_thibedeau 11y ago
      VB.NET is a reskin of C#.
      - protomyth 11y ago
        Yeah, but in Ada's case it would look like you're going the other way (to { }). [edit: folks really shouldn't down vote kevin_thibedeau, he is correct after all]
    - nickpsecurity 11y ago
      I considered making a variant of Go or Pascal that enforced Ada's safety properties for common operations by default without anything else in it. Turn it off on a per-module basis still with type or interface checks. Result might get a bit more uptake.
  - ufo 11y ago
    If you want to have a language be a "C killer" for secure software you need to put a lot of emphasis on simplicity and portability, which are not strengths of Rust. I think that if you get rid of raw pointers and type casts, you are able to avoid most undefined behavior with dynamic checks: bounds checking, tagged unions, etc. The notable exception is concurrency bugs, which are hard to avoid without a restrictive static analyzer, but I think there is an argument to be made that perhaps the kind of code mr Bernstein wants to write is better off with "boring" sequential algorithms.
- deleted 11y ago
  [deleted]
- Animats 11y ago
  It's too bad Pascal is dead. Wirth's original recursive-descent Pascal compiler was utterly straightforward. It produced mediocre, predictable code. I have hope for Rust, but in terms of complexity, it seems to be picking up where C++ left off.
  - atemerev 11y ago
    It is not dead. You can still take FreePascal and write some security conscious code with it, to be linked with other parts of your application. The compiler works. There is also Lua. And Haskell and Ocaml compilers for FP people. These languages are easily linkable with C/C++ codebases.
  - kibwen 11y ago
    I'm not sure where you got the impression that "Rust is picking up where C++ left off". Rust is a far, far, far smaller language than C++, and has benefited from hindsight in its design in a way that C++ cannot without massively breaking backwards compatibility.
  - nickpsecurity 11y ago
    What did you think about Modula-3? I like how it had much of what a C++ programmer might want in a cleaner, safer, faster-to-compile way. Most Modula and Oberon derived languages had little to no undefined behavior outside the Unsafe modules. All were used to write OS's & other low-level code. Optional GC where it was tolerable. IIRC, Modula-3 also had verification done on its standard modules to catch certain errors. Some of the design attributes might be worth resurrecting in another project. Minus the capitalized keywords. :)
    - Animats 11y ago
      I wrote much of an OS in Modula I. I wrote robotics code in Modula II. I didn't do much with Modula III, but I knew some of the people behind it at DEC SRL, before Compaq bought DEC and shut down research. I wish computing had gone that way, instead of C/C++. Everything would be less buggy.
- IvyMike 11y ago
  > Writing secure software in C is just next to impossible. This is true. And it was true when I learned C, and was told almost the same exact thing on the second day of class. In 1992. Maybe eventually we'll collectively move to something safer, but at this point I've stopped holding my breath. DJB's proposal seems pretty reasonable in the interim.
- rosser 11y ago
  Then how do you explain the fact that some of the most secure pieces of software ever written were written in C? (I'm thinking of, e.g., Qmail — and pretty much everything else DJB has ever written — and OpenBSD, just to name a couple off the top of my head.) I would also qualify your statement that "performance is not everything" to say that performance isn't always everything. Writing a kernel in an interpreted language probably isn't the best idea, for example.
  - admax88q 11y ago
    Because C has a lot of momentum, and interoperability, thus lots of security software was written. This software isn't secure because of C but despite of it, and has had a long list of security vulnerabilities that a safer language would have protected us from.
    - mayank 11y ago
      > and has had a long list of security vulnerabilities that a safer language would have protected us from. Care to point to the "long list of security vulnerabilities" in qmail?
      - oldmanjay 11y ago
        I'm very interested to know what point you believe you are making here. I ask because it is fairly clear you aren't addressing the topic of the thread and I wonder if you believe you are.
  - gherkin0 11y ago
    > Then how do you explain the fact that some of the most secure pieces of software ever written were written in C? (I'm thinking of, e.g., Qmail — and pretty much everything else DJB has ever written — and OpenBSD, just to name a couple off the top of my head.) Unusual amounts of manual effort by unusually skilled people, probably using unusually security-focused development processes?
    - 11y ago
- chillingeffect 11y ago
  Please don't spread unnecessary fear. ("deep magic"). C is as secure as developers take their time to implement. A fantastic amount of dangerous stuff currently runs on C. C has several substandards, such as MISRA and JSF, to aid developers in writing safe C, eliminating arguments about "well-defined." A good start is bounds checking and following MISRA. Languages like Rust make it cheaper, but security never comes for free.
  - atemerev 11y ago
    Indeed, security never comes for free, but costs can vary tremendously. In case of C, there are layers of additional rulesets (not enforced by compiler), scrutiny and coding standards even for basic things. I do not subscribe to formal verification crowd claims, but some basic things can make great improvements. E.g. immutability by default.
- spc476 11y ago
  Please define which assembly you are talking about. Older CPUs didn't trap on undefined opcodes (I'm talking things like the 8080, Z80, 6502, 6809, etc). And even on a more modern CPU (say, the x86 line) I see that the condition flags after a DIV are all undefined (Z, C, S, O, all undefined; also, MUL also leaves the flags undefined). But yes, assembly languages tend to be more defined than C.
sbierwagen 11y ago
CompCert?
- rfw 11y ago
  I believe Compcert still translates programs with undefined behavior and doesn't really do anything about it; or rather, the behavior is defined but based on whatever the Compcert authors define the behavior to be.
e28eta 11y ago
My understanding was that with optimizations off, clang/gcc were pretty "boring". Is a whole new compiler really required? Personally, to fix these problems I'd lean toward finding a new language instead of a new compiler.
- sanxiyn 11y ago
  For example, OP suggests zeroing uninitialized variables. Neither clang nor gcc with optimizations off do that.
sz4kerto 11y ago
" Claim that all we need is for some particular "undefined"-catching tool to be widely used. In fact, these tools are full of false positives and false negatives; at best they catch a few limited types of "undefined" behavior, not changing the big picture." Question: why can't existing C compilers clearly indicate when they encounter undefined behavior?
- pdkl95 11y ago
  While it is possible (in theory) to catch some types of undefined behavior, some of it can be very subtle, and it shows up in the C Standard far more often than most people think. https://www.securecoding.cert.org/confluence/display/c/CC.+Undefined+Behavior#CC.UndefinedBehavior-ub_1 https://www.securecoding.cert.org/confluence/display/c/CC.+U...
- MereInterest 11y ago
  Undefined behavior is primarily something that that has effects, rather than at compile time. For example, taking the value of an uninitialized local variable. int main() { int i; if(some_big_complicated_function()){ i = 0; } printf("%d\n",i); } Is this undefined behavior? The compiler can't be sure, because it can't know whether `some_big_complicated_function()` returns true or not. Therefore, it would need to be a runtime check, rather than a compile time check. But runtime checks are expensive, and so it typically isn't done. It would be entirely allowed by the standard to give such a warning at runtime, but the resulting program would go much, much slower due to all the checks.
  - sz4kerto 11y ago
    This would certainly worth a warning (at some warning level) exactly because what you say -- the compiler is not sure that the behavior is fully defined.
    - MereInterest 11y ago
      I wasn't sure whether or not it would print out a warning, so I tested it. With g++ 4.8.4, there is no warning printed, even with `-Wall -Wextra -pedantic`.
deleted 11y ago
[deleted]
throwaway999888 11y ago
Clearly what you want is something other than C.
- tormeh 11y ago
  Ada, and recently Rust, for example. People trying to improve C are sad to watch. People use C because they have an irrational belief in their own faultlessness. Same reason people are afraid of flying but not driving. Selling a correctness-enhancing tool to these people is an impossible task.
  - nraynaud 11y ago
    I recently tried to use Ada, that ended in sadness. I couldn't try Rust, because it doesn't (didn't?) run on arm-none-eabi.
    - steveklabnik 11y ago
      Rust should work on `arm-none-eabi` to my knowledge. We don't provide packages out of the box, but people have done it. You need to set up the right target specification, and then it should work.
      - nraynaud 11y ago
        Possible, I don't remember the details. It might also be that setting up a cross compiling toolchain by hand was not exactly how I intended my first contact with a new language be (I had set up a cross compiling GCC, then a cross compiling Ada that ended up in a linux simulator, I might have lazied on the rust test).
        steveklabnik 11y ago
        Yeah, it's not ideal. We are actively working on making it easier to do these kinds of things; it's not well documented, to be honest.
  - 11y ago
imron 11y ago
Surely this is where things like __attribute__ ((optnone)) and #pragma optimize("", off) come in useful.
theseoafs 11y ago
The compiler for this language will be too slow to be useful. Contrary to this author's belief, crypto people really do care about performance. Anyway, what the author wants isn't a "boring C" compiler, it's a compiler for a new language that doesn't actually have undefined behavior.
- geertj 11y ago
  You realize the author is Dan Bernstein, right?
- guelo 11y ago
  From the article: > * Claim that a boring C compiler can't possibly support the desired system _performance_. Even if this were true (which I very much doubt), why would it be more important than system _correctness_?
  - theseoafs 11y ago
    I read the article. I don't know if the author has considered the implications of what he wrote. What we're talking about is: * Treating all memory accesses as if they were defined * No dead code elimination * Treating all pointers as if they're volatile This adds up to a lot of overhead.
  - 0xcde4c3db 11y ago
    It can't be taken for granted that performance and correctness are separate concerns. Even leaving aside the obvious category of realtime applications, performance issues can create DoS vulnerabilities. edit: That being said, a boringcc might end up being less susceptible to these vulnerabilities in practice, since the performance of the emitted code might be more predictable. I'm sure djb has considered this. But in general I think it's often a mistake to think of performance as an implementation detail rather than as a deliberate feature.
tmalsburg2 11y ago
It sounds as if the root of the problem is the C standard. Wouldn't it be better to fix that instead of creating a shadow standard defined by the implementation of this boring compiler? I also object to the presupposition that there is secure software and non-secure software. In most practical cases, it is impossibly hard to tell in which class an application falls, so it would be better not to make that distinction at all.
- beefhash 11y ago
  >It sounds as if the root of the problem is the C standard. Wouldn't it be better to fix that Isn't the reason for a lot of undefined behavior either in the name of performance through optimization and assumptions, portability because the implementation can just do whatever the underlying platform natively does or the committee just couldn't agree on it? All three of those scenarios appear inherently unfixable to me without a separate sub-standard that intentionally clamps down on performance/ease of portability and actually gets the committee to agree.
  - tmalsburg2 11y ago
    I don't know the reasons for why so many things are undefined in C. However, if a program does one thing when compiled with compiler A and another thing when compiled with compiler B, how does that help portability? To my perhaps naive mind this makes portability harder to achieve not easier.
    - pjc50 11y ago
      A classic example of trying to define behaviour regardless of what all the different hardware does is Java choosing IEEE754 floating point. Which isn't natively supported on x86. https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf If C had standardised to the idiosyncracies of the PDP-11, it might have held back its development.
      - jey 11y ago
tacostakohashi 11y ago
I've always thought that there would be more value in exactly the opposite - a pathological C compiler (or C++ compiler, POSIX operating system, etc.). The pathological compiler would always implement undefined behavior in the most surprising, unpredictable manner - e.g., uninitialized memory would have a properly randomized value that changes from one run to the next, overflows would result in random values, etc. That way, any programs that are made to work on the pathological implementation are largely guaranteed to rely only on standardized behavior, and work on any other standards compliant implementation.
- 1ris 11y ago
  But why? That's a way more complicated and a way less usefull.
  - MereInterest 11y ago
    Because if you have undefined behavior, you want your program to crash and burn as quickly as possible, so that you can remove the undefined behavior. What seems to be proposed by Dan Bernstein is not so much a "boring C compiler", but rather a superset of C which does not have any undefined or unspecified behavior.
    - kazinator 11y ago
      Sometimes if you have undefined behavior, you want the documented extension to happen. Or you want the de facto widely understood behavior to happen which has happened on every compiler you've used in 30 years: at least every compiler that was for two's complement matchines. Or every compiler on which pointers to different data types were of the same size. And so on. You also don't want to be burned by code that is relying on a common extension, when that is ported. There is a word for the compiler approach of "we're going to optimize this based on the assumption that the program is not relying on a common extension that is technically UB". That word is: malpractice. For instance, it is not unusual for C code to be targetting machines in which all objects are in the same address space, such that pointers are internally like binary numbers, allowing pointers to different objects to be compared for inequality: ptr1 < ptr2 /* does ptr1 point to a lower address than ptr2 in the one big address space? / Programmers expect this work consistently with the address structure of the machine. They don't want the expression to be somehow wrongly optimized based on the assumption that ptr1 and ptr2 must point to the same object. Implementations must honor requirements beyond those of ISO C. Such as, for instance, behaviors that those same compilers* used to define historically in their past revisions. If GCC has behaved a certain way for 25 years, and some code has come to depend on that, and then that is suddenly taken away, such that entire GNU/Linux distros continue to compile, but break in random places because of that detail, the fault for the regression lies 105% with the doofus who made the GCC commit. Even if that behavior is not defined by ISO C.
pool 11y ago
How far are we from having the will and ability for it to be common to use safer languages than C/C++/Java/etc.? A long time ago, it didn't seem pragmatic to write in, say, ocaml, because no one was going to feel like building or even installing the compiler in order to try your dumb software, but now that everything is about packages from distributions... Today it feels strange to apply security updates for necessary software where the announcements list "Multiple memory safety errors, integer overflows, use-after-frees..."
jbandela1 11y ago
This is actually something the ISO C++ committee is looking at. See https://isocpp.org/std/the-committee https://isocpp.org/std/the-committee - SG12 An example of this is P0145R0 which specifies the previously unspecified order of evaluation. See https://isocpp.org/blog/2015/11/kona-standards-meeting-trip-report https://isocpp.org/blog/2015/11/kona-standards-meeting-trip-... Apparently, the performance difference is expected to be zero on currently popular architectures and around 1% on older architectures.
makecheck 11y ago
One of the bases for security is the age of a program. All other things being equal, the implementation that has been around the longest is expected to have fewer issues than the shiny, new thing (because the old program is well-understood, has been patched to death, considers a thousand corner cases, ran on more hardware, more time passed without any issues, etc.). As such, changing away from well-known compilers and language behaviors may not be the wisest thing for security. On the other hand, extending existing compilers to restrict behavior seems reasonable. Also, by all means document the hell out of the things that are not well-defined.
AndyMcConachie 11y ago
Programmers always want new and more clever constructs in their code, and there is always the next feature that will optimize inner loops 10% better. This is human nature apparently, so don't put too much faith in a boring anything gaining traction. And for those of you courting Rust, remember there are two kinds of programming languages in the world: those that no one uses, and those that are known to be terrible.
eliteraspberrie 11y ago
It exists. It's called CompCert. https://github.com/AbsInt/CompCert https://github.com/AbsInt/CompCert It's free for non-commercial use. I've used it for several months now to build things like Tor. I haven't noticed any disadvantage compared to GCC or Clang.
- bpicolo 11y ago
  > the generated assembly code is formally guaranteed to behave as prescribed by the semantics of the source C code that suggests to me you could still hit C's undefined behaviors. It's only guaranteeing the semantic equivalence between source and output
- ufo 11y ago
  I don't think compcert avoids undefined behavior pitfalls. Its more of a way to protect yourself from compiler bugs.
  - scott_s 11y ago
    Agreed - I read through their website, and I saw no mention of eliminating undefined behavior.
  - nickpsecurity 11y ago
    It's a verified subset of the C standard rather than all behaviors of all compilers. There's some semantics and work covering a lot of that already that just needs to be integrated into CompCert or static analyzers. Likewise, there's static analyzers that already cover it too but not formally verified. Here's an example of investigating that with Compcert: http://robbertkrebbers.nl/research/articles/compcert_formalin.pdf http://robbertkrebbers.nl/research/articles/compcert_formali... Here's a semantics that looks for flaws and undefined behavior: https://www.ideals.illinois.edu/bitstream/handle/2142/34297/Ellison_Charles.pdf?sequence=1 https://www.ideals.illinois.edu/bitstream/handle/2142/34297/...
- nickpsecurity 11y ago
  You beat me to it. Unlike the other commenters, I see that you're describing it as a robust, predictable start to what Bernstein describes. Undefined behavior and other problems can be caught by extensions to the compiler or via static analysis tools like an Astree Analyzer knockoff. The fact that they'll be written in Ocaml (or SML) will make them more reliable. A good choice and what I've promoted here so far. Additionally, the clean passes are easier to bootstrap for those that fear subversion of their compiler. Can use whatever you want on whatever machine to implement it. So long as code matches spec, the output should be the same. That implementation compiles the other compilers, verifiers, whatever. You might also enjoy that people have already extended CompCert work and others for developments that push in the Boring, Safe/Secure, C compiler direction. Here's some: CakeML - Compile CompCert, etc with this https://cakeml.org/ https://cakeml.org/ Verasco - verified, CompCert-based, static analyzer for C http://compcert.inria.fr/verasco/ http://compcert.inria.fr/verasco/ Verification of LLVM transformations later work (eg Softbound or Code-pointer Integrity clone) http://repository.upenn.edu/dissertations/AAI3592852/ http://repository.upenn.edu/dissertations/AAI3592852/ Extension of CompCert for verified, WCET analysis https://hal.inria.fr/hal-01087194 https://hal.inria.fr/hal-01087194 So, lots of good stuff being done by a subset that focus on right design methods with the right tools. And they're getting incredible results. Surprise! :)
deleted 11y ago
[deleted]
kazinator 11y ago
Some small steps could have been made years ago. For instance, getting rid of unspecified order of evaluation. Or defining a dialect of ISO C in which evaluation is strictly left to right, and making support for that optional (so then your local GCC or whatever can have -fstrict-iso-eval to turn it on). Another example of "small stuff": catching negative values being passed into the <ctype.h> functions. Also, all the instances in the standard library where a null pointer invokes undefined behavior could be required to abort the program. Some undefined behaviors cannot be made defined without a lot of overhead (like catching all out of bounds pointers). However, there is a lot of "low hanging fruit" undefined behavior where things can be improved.
petke 11y ago
> I'd like to see a free C compiler that clearly defines, and permanently commits to, carefully designed semantics for everything that's labelled "undefined" or "unspecified" or "implementation-defined" in the C "standard". That wouldn't just be a new compiler. That would be a new language. I dont think there are any short cuts to fix C. C++ is doing something somewhat similar with the "C++ Core Guideline" rules, which are designed to be checkable by static analysis tool. The long term goal is that eventually C++ will become a safe subset, when all the error-prone parts of the language have replacements. > "Following the rules will lead to code that is statically type safe, has no resource leaks, and catches many more programming logic errors than is common in code today. And it will run fast - you can afford to do things right." https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...
- geocar 11y ago
  > That would be a new language I don't see why it wouldn't be a valid implementation of C by simply defining things that C doesn't fully define. There's nothing about the definition of "undefined behaviour" that prevents you from defining it in your implementation.
  - steveklabnik 11y ago
    Because then you are relying on a specification that's not in the spec. A different interpretation means that your code no longer works. Therefore, you've effectively created a new language.
    - geocar 11y ago
      You argue if GCC and MSVC document their current undefined behaviour, they are creating new languages?
      - steveklabnik 11y ago
        Not if they document it. But if you write a program that relies on those behaviors, you don't have C anymore, you have "GCC-C" or "MSVC-C". I'm also not saying that's inherently a bad thing.
xorblurb 11y ago
At this point you pretty much want an alternative language more than a compiler: one that looks like C, but is completely defined. And this makes kinda sense, yet is IMO still quite dangerous (less than the actual situation but still too much) 1/ There are tons of people who will say: "this project is written in C" instead of the reality that is that this project is written in boringC, (actually probably even most of the projects authors will) and will for ex. compile with a regular C compiler instead of a boringC one, maybe for performance improvement... 2/ You just can't take C, even with a completely defined semantics, and call it a day for secure programming. It would still be filled with crazy features that are intrinsically impossible to render secure (everything about C "arrays" comes to mind) So let's take this idea to its real destination and rewrite secure projects using a really secure programming language, instead of just extending the current mess for 30 more years.
- Gracana 11y ago
  > At this point you pretty much want an alternative language more than a compiler: one that looks like C, but is completely defined. A "Friendly C" dialect has been proposed: http://blog.regehr.org/archives/1180 http://blog.regehr.org/archives/1180
- aikah 11y ago
  https://cyclone.thelanguage.org/ https://cyclone.thelanguage.org/ ? kind of ignored by the mass and unsupported now, yet this is exactly the kind of project the developer community needs... a language that is minimalist, yet with enough feature to allow some expressiveness and speed.
  - dikaiosune 11y ago
    "(Several of Cyclone's ideas have made their way into Rust.)"
- Terr_ 11y ago
  So... Java? :P
- fbbbbb 11y ago
dikaiosune 11y ago
I'm someone currently in the process of drinking the Rust kool-aid, in part because the goals of the language are very closely aligned with what Bernstein proposes here. Further, because of the memory model, the compiler can produce quite competitive performance while still preventing the execution of any undefined behavior (related to his last strawman in the bullet list). I get that even if Rust offers a panacea to the issues he mentions (which it may or may not, I'm no expert), there's still an awful lot of C code that would need to be deprecated and rewritten. But surely there'd be value in efforts to get away from a language that has such a loose standard?
- wspeirs 11y ago
  I wonder how hard it would be to create a C -> Rust transpiler. The output would probably be God-awful, but it could be a start to moving away from C towards Rust.
  - steveklabnik 11y ago
    Yes, the problem with doing this is that Rust really imposes certain kinds of design constraints that makes this hard. Like, you COULD, but you'd make a lot of use of unsafe, in a way that's not idiomatic. Part of the problem here is that the program may rely on subtleties that make the translation hard. I've seen experienced C or C++ people come into the Rust IRC and ask why certain things aren't allowed, it's often due their code relying on certain aspects of the looser model. That said, it's very possible to go unsafe first and then refactor. It just might not be easy. Another way of making the transition is to pick a component and only re-write it. This is the approach Firefox is taking, for example.
  - mcguire 11y ago
    A guess: very, very difficult. Writing code to satisfy Rust's checking is not much harder than writing C[1], but converting arbitrary C to satisfy Rust is something completely different. [1] An' I say it as shouldn't. C is my natural language.
- ethbro 11y ago
  It's probably going to be a hard row to hoe to get any code that doesn't always visibly break out of circulation. That said, I think that's why there's huge value in moving to languages where you do break things more visibly. Ultimately it comes down to time, and "It looks like it works but might do something wrong sometime" is a weaker argument to delay release than "It will not compile/run".
raverbashing 11y ago
I don't think this can be done with C. Really - Remove all possibilities of memory aliasing. Only exception might be when reading from devices/network (some kind of boxing, unboxing) - All array accesses are bounds checked. - Checked conversion between int/unsigned int and different sized items. Converting a char into an unsigned int? Be explicit in what you want to do - No pointers "flying around", unbound to types. Have reference-counted objects at least. Allow a way of passing a sub-array without copying/with read only permissions. - Remove "gotchas", like ordering of statements, assignment on comparison (unless it's obviously correct), no "compiler dependent" behaviour, no "undefined behaviour" (unless it's machine specific)
- sleepydog 11y ago
  You are confusing unsafe behavior with undefined behavior. He is not suggesting that a C compiler should prevent the above mistakes. He's suggesting that it should document what code the compiler will emit in such cases. For example, - reading past the end of an array will always segfault the program immediately. or reading past the end of an array will return (arrayType)0 - signed integer types must overflow according to two's complement. A compiler may not assume that, for instance, (int)x < (int)x + 1 is always true. - uninitialized variables are zeroed out. That is, `int *x;` always produces a null pointer, `char y` always produces '0' - A compiler may not remove any statements where it determines such an optimization would not change the behavior of the program. See http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buf... Such a compiler would accept and compile any C program, and the programmer (and any static analysis tools) would have a better idea of the behavior of the output program.
  - timmytokyo 11y ago
    > reading past the end of an array will always segfault the program immediately. or reading past the end of an array will return (arrayType)0 How would this work when you pass the array (i.e., the pointer) to a function? In order to do what you describe, every pointer would need to include bounds information. This would break the C ABI, so your code would no longer link with existing C libraries.
rwmj 11y ago
gcc -O0 Would that do?
- sanxiyn 11y ago
  gcc -O0 does not zero uninitialized variables, which is among things explicitly suggested by OP.
lisper 11y ago
It has always seemed to me that the C community (and to an ever greater extent the C++ community) takes particular pride in making it as difficult as possible to write correct code. It provides bragging rights and job security for those select few who have taken the time to master all of the arcana of the language. Personally, I tend to think that life is too short for that sort of thing. That's why I choose Lisp, at least when it's up to me.
- nickpsecurity 11y ago
  Surprised being a LISPer you haven't read Gabriel's Worse is Better essays on the topic. Convincingly argue the reason behind the uptake of certain tech is they do just enough of a good job to work and for a lot of people. Then they get extended gradually in direction of what they should've been. C and UNIX were perfect examples of this on crappy hardware of 70's-80's. By 90's, it was legacy effect in play for various reasons. Still is. Throw in organizations competing for money or ideology to get a bunch of implementations of a language that was shit to begin with. So, it was more the opposite of what you claim: too many groups trying to make things convenient for themselves in the short-run. LISPers did something similar with their individual messes getting merged into Common LISP. Those doing it clean and sensible on the right hardware were always the outliers.
  - lisper 11y ago
    I have read "Worse is better." The point I was trying to make was different: > Then they get extended gradually in direction of what they should've been. Yes, this is the part I'm disputing. It seems to me that C and C++ are not being extended in "the direction they should have been" but rather in a direction that is beneficial mainly to a small incumbent minority. Everyone agrees that security is important, and yet it is virtually impossible to write secure code in C or C++, not because it would violate the laws of physics or some mathematical theorem, but because both the standards and the implementations are hopelessly brain-damaged.
    - nickpsecurity 11y ago
Arnt 11y ago
I don't understand this. He's not asking for a -wundefined or -ansi -pedantic, he's asking for definitions for everything -ansi -pedantic rejects, and a new compiler. Hard work. Which of those cases are that desirable?
tikums 11y ago
OpenBSD (+LibreSSL) folks have been uncomfortable with unpredictable compiler optimizations for quite a while too. Here's a previous discussion on compilers in OpenBSD: https://news.ycombinator.com/item?id=9322259 https://news.ycombinator.com/item?id=9322259
ape4 11y ago
In the past initializing variables to zero might have been too slow. But not now.
gdwatson 11y ago
I think the biggest problem with undefined behavior in C is that it is taken to justify violation of C's customary semantics[1] on platforms where it isn't necessary. Many types of undefined behavior in the C standard are essentially dispensations for Lisp machines or one's complement CPUs. It's very surprising to see them invoked by the optimizer on more conventional platforms. It might not satisfy djb, but a standard for these behaviors alone would be great progress.[2] [1] http://arcanesentiment.blogspot.com/2014/12/customary-semantics.html http://arcanesentiment.blogspot.com/2014/12/customary-semant... [2] Posix doesn't really address this, sadly. The committee seems to have avoided standardizing byte size until an unforeseen interaction with a later C standard pushed it into a corner.
pcwalton 11y ago
Required reading here is Chris Lattner's "What Every C/C++ Programmer Should Know About Undefined Behavior", which goes into good detail as to why undefined behavior is important for performance, sometimes very important: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html?m=1 http://blog.llvm.org/2011/05/what-every-c-programmer-should-... Could you make a new language that doesn't depend on UB for performance? Sure, quite possibly. Is C that language? Probably not.
cbd1984 11y ago
He says this: > Claim that a boring C compiler can't possibly support the desired system _performance_. Even if this were true (which I very much doubt), why would it be more important than system _correctness_? If correctness is more important than performance, C is the wrong language. Consider a language with, at the very least, garbage-collection built into the core, as memory allocation and usage errors are the base of a lot of what goes badly wrong with C programs. In short, his notion of "Boring C" seems to be Haskell or Common Lisp.
- nickpsecurity 11y ago
  Those languages are incredibly complicated. I'd go with Modula-2 or Pascal as a boring C candidate.
  - cbd1984 11y ago
    I'm not sure that either of those languages are usually GC'd, but they'd be good choices, too, if they were. (I just don't like Wirth languages. I don't like their type systems and I don't like the definition-before-use requirement. They're better than C, in some respects, but worse in others, and the net isn't always an improvement.)
    - nickpsecurity 11y ago
      All of Wirth's after Modula-2 were GC'd. C programmers would reject a GC'd language, though. So, I stayed with originals that enhanced safety, predictability, readability, and so on while capable of low-level, fast stuff. Wirth's stuff and industrial variants of it often let you do unsafe stuff in dedicated modules for that so you could turn off GC or checks. Plus they wrote OS's in Modula-2 and Pascal on ancient hardware. Whole platforms. So, they're proven for the use case C programmers often say you need their language for.
      - cbd1984 11y ago
        Whole OSes were written in Lisp, too, as well as other GC'd programming languages (Smalltalk, Mesa, perhaps a few others), and they ran on rather underpowered hardware by modern standards as well. I get your point, but having to check for allocation failure is a substantial source of bugs in non-trivial programming without GC, and having to manually allocate RAM makes string handling, for example, orders of magnitude less safe than it is in a language with GC.
cornholio 11y ago
Event the greatest brains can fart, this proposal can be safely ignored.
chris_wot 11y ago
Is there any list of undefined behaviours in C? Other than the standard.
aplorbust 11y ago
Actual URL, accessible without Javascript: https://groups.google.com/forum/message/raw?msg=boring-crypto/48qa1KWignU/o8GGp2K1DAAJ https://groups.google.com/forum/message/raw?msg=boring-crypt...
Too 11y ago
I think the problem is not the language standard per se, it's the education and historically sloppy compilers. It's a bit like W3Schools for javascript teaching everybody all the worst practices, that's not a problem with the javascript language itself. I mean really, what are the most common cases of undefined behavior that cause real problems in the world? (Reading past array bounds or trying to dereference uninitialized variables are such obvious errors that i don't categorize them as 'unexpected undefined behavior', the later is caught by most compilers and static analyzers today anyway. Though i agree having the length known in arrays would greatly help.) Any time i see some raging blog post about how undefined behavior ruined someones day it's 99% of the time because they one way or another were trying to cast a pointer of one type to a pointer of another type, like thinking they could read the first 16 bits of an int* by casting it to a short* first. The root cause of this is because people think that pointers are addresses to a sequential grid of NAND-gates on your DDR4-stick, when they in fact are abstract references to a variable that can carry a value. This comes from school, where the way teachers show you how pointers work is exactly like that. Tutorials on the internet also say that you can do smart tricks with pointers just to teach you and then people think that's how you should do it. When compiling in debug-mode this is how the memory is usually aligned so things still work anyway. Historically compilers didn't warn about this either and things still worked so many people got used to writing things this way and today those people are the "experienced C gurus" in companies that pass down this misbehavior to their younger peers. Overflow on signed arithmetic is another of the common pitfalls but that's life if you code for embedded, the alternative solution that "just works" would be to do all arithmetic in a BigInt type as python but that clearly isn't viable on embedded, or to do everything in floating point but that has its own pitfalls.

274 points by DCKing 11y ago

Non mobile link: https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWignU/o8GGp2K1DAAJ https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWi...

atemerev 11y ago

Writing secure software in C is just next to impossible. Memory corruption, arbitrary pointers, deep magic. Even pure assembly is more secure, as everything is explicit and well-defined there. There are lots of safer languages than C, and performance is not everything.

stcredzero 11y ago
Is there a language that has powerful tools for dealing with timing attacks and can facilitate correctness proofs?
- TheMagicHorsey 11y ago
  Ocaml
  - sbierwagen 11y ago
    Isn't ocaml garbage collected? Won't the GC pauses leak timing info?
    - bmm6o 11y ago
      If you are generating different amounts of garbage depending on your secrets, that's already showing up as leaked timing information.
- nickpsecurity 11y ago
  You can deal with timing attacks using Kemmerer's Shared Resource Matrix to find those and other covert channels. Then, you just look at the bandwidth of them along with design alternatives that don't leak timing. You can sometimes make the timing predictable or non-data-driven at interface level to nullify that. An example comes from military-grade stuff way back that transmitted in fixed-length, fixed-rate packets. Markus Ottela added that to Tinfoil Chat at my recommendation. Far as correctness proofs, a combo of Ada 2012 with Design-by-Contract and SPARK is probably best at that. Far as app level, just program in a functional style with abstract state machines. Make it more concrete and detailed, replicating analysis or tests at each level. Eventually, you run the code checkers on the result and throw in some fuzz testing. No easy route, though, that I'm aware. Highly assured software and system design is a painstaking process. Avoiding risky stuff, defining all success/fail traces, failing safe, and static analysis to preempt bugs are the strongest route. Plus source-to-object correspondence to be sure compiler didn't eliminate safety checks or introduce problems. Spark Pro supports that.
nine_k 11y ago
I heard Rust can be pretty efficient? OTOH a heavily restricted C, closer to assembly than to Rust, could still be useful: it would be close to the metal but still preserve portability. Making such a language a strict subset of C would additionally help early porting efforts.
- protomyth 11y ago
  Ada can be amazingly efficient. It just isn't cool. I've often wondered what would happen if someone pulled a reskining of the language (ala VB.NET >> C#) what would happen.
  - kevin_thibedeau 11y ago
    VB.NET is a reskin of C#.
    - protomyth 11y ago
      Yeah, but in Ada's case it would look like you're going the other way (to { }). [edit: folks really shouldn't down vote kevin_thibedeau, he is correct after all]
  - nickpsecurity 11y ago
    I considered making a variant of Go or Pascal that enforced Ada's safety properties for common operations by default without anything else in it. Turn it off on a per-module basis still with type or interface checks. Result might get a bit more uptake.
- ufo 11y ago
  If you want to have a language be a "C killer" for secure software you need to put a lot of emphasis on simplicity and portability, which are not strengths of Rust. I think that if you get rid of raw pointers and type casts, you are able to avoid most undefined behavior with dynamic checks: bounds checking, tagged unions, etc. The notable exception is concurrency bugs, which are hard to avoid without a restrictive static analyzer, but I think there is an argument to be made that perhaps the kind of code mr Bernstein wants to write is better off with "boring" sequential algorithms.
deleted 11y ago
[deleted]
Animats 11y ago
It's too bad Pascal is dead. Wirth's original recursive-descent Pascal compiler was utterly straightforward. It produced mediocre, predictable code. I have hope for Rust, but in terms of complexity, it seems to be picking up where C++ left off.
- atemerev 11y ago
  It is not dead. You can still take FreePascal and write some security conscious code with it, to be linked with other parts of your application. The compiler works. There is also Lua. And Haskell and Ocaml compilers for FP people. These languages are easily linkable with C/C++ codebases.
- kibwen 11y ago
  I'm not sure where you got the impression that "Rust is picking up where C++ left off". Rust is a far, far, far smaller language than C++, and has benefited from hindsight in its design in a way that C++ cannot without massively breaking backwards compatibility.
- nickpsecurity 11y ago
  What did you think about Modula-3? I like how it had much of what a C++ programmer might want in a cleaner, safer, faster-to-compile way. Most Modula and Oberon derived languages had little to no undefined behavior outside the Unsafe modules. All were used to write OS's & other low-level code. Optional GC where it was tolerable. IIRC, Modula-3 also had verification done on its standard modules to catch certain errors. Some of the design attributes might be worth resurrecting in another project. Minus the capitalized keywords. :)
  - Animats 11y ago
    I wrote much of an OS in Modula I. I wrote robotics code in Modula II. I didn't do much with Modula III, but I knew some of the people behind it at DEC SRL, before Compaq bought DEC and shut down research. I wish computing had gone that way, instead of C/C++. Everything would be less buggy.
IvyMike 11y ago
> Writing secure software in C is just next to impossible. This is true. And it was true when I learned C, and was told almost the same exact thing on the second day of class. In 1992. Maybe eventually we'll collectively move to something safer, but at this point I've stopped holding my breath. DJB's proposal seems pretty reasonable in the interim.
rosser 11y ago
Then how do you explain the fact that some of the most secure pieces of software ever written were written in C? (I'm thinking of, e.g., Qmail — and pretty much everything else DJB has ever written — and OpenBSD, just to name a couple off the top of my head.) I would also qualify your statement that "performance is not everything" to say that performance isn't always everything. Writing a kernel in an interpreted language probably isn't the best idea, for example.
- admax88q 11y ago
  Because C has a lot of momentum, and interoperability, thus lots of security software was written. This software isn't secure because of C but despite of it, and has had a long list of security vulnerabilities that a safer language would have protected us from.
  - mayank 11y ago
    > and has had a long list of security vulnerabilities that a safer language would have protected us from. Care to point to the "long list of security vulnerabilities" in qmail?
    - oldmanjay 11y ago
      I'm very interested to know what point you believe you are making here. I ask because it is fairly clear you aren't addressing the topic of the thread and I wonder if you believe you are.
- gherkin0 11y ago
  > Then how do you explain the fact that some of the most secure pieces of software ever written were written in C? (I'm thinking of, e.g., Qmail — and pretty much everything else DJB has ever written — and OpenBSD, just to name a couple off the top of my head.) Unusual amounts of manual effort by unusually skilled people, probably using unusually security-focused development processes?
  - 11y ago
chillingeffect 11y ago
Please don't spread unnecessary fear. ("deep magic"). C is as secure as developers take their time to implement. A fantastic amount of dangerous stuff currently runs on C. C has several substandards, such as MISRA and JSF, to aid developers in writing safe C, eliminating arguments about "well-defined." A good start is bounds checking and following MISRA. Languages like Rust make it cheaper, but security never comes for free.
- atemerev 11y ago
  Indeed, security never comes for free, but costs can vary tremendously. In case of C, there are layers of additional rulesets (not enforced by compiler), scrutiny and coding standards even for basic things. I do not subscribe to formal verification crowd claims, but some basic things can make great improvements. E.g. immutability by default.
spc476 11y ago
Please define which assembly you are talking about. Older CPUs didn't trap on undefined opcodes (I'm talking things like the 8080, Z80, 6502, 6809, etc). And even on a more modern CPU (say, the x86 line) I see that the condition flags after a DIV are all undefined (Z, C, S, O, all undefined; also, MUL also leaves the flags undefined). But yes, assembly languages tend to be more defined than C.

sbierwagen 11y ago

CompCert?

rfw 11y ago
I believe Compcert still translates programs with undefined behavior and doesn't really do anything about it; or rather, the behavior is defined but based on whatever the Compcert authors define the behavior to be.

e28eta 11y ago

My understanding was that with optimizations off, clang/gcc were pretty "boring". Is a whole new compiler really required? Personally, to fix these problems I'd lean toward finding a new language instead of a new compiler.

sanxiyn 11y ago
For example, OP suggests zeroing uninitialized variables. Neither clang nor gcc with optimizations off do that.

sz4kerto 11y ago

" Claim that all we need is for some particular "undefined"-catching tool to be widely used. In fact, these tools are full of false positives and false negatives; at best they catch a few limited types of "undefined" behavior, not changing the big picture." Question: why can't existing C compilers clearly indicate when they encounter undefined behavior?

pdkl95 11y ago
While it is possible (in theory) to catch some types of undefined behavior, some of it can be very subtle, and it shows up in the C Standard far more often than most people think. https://www.securecoding.cert.org/confluence/display/c/CC.+Undefined+Behavior#CC.UndefinedBehavior-ub_1 https://www.securecoding.cert.org/confluence/display/c/CC.+U...
MereInterest 11y ago
Undefined behavior is primarily something that that has effects, rather than at compile time. For example, taking the value of an uninitialized local variable. int main() { int i; if(some_big_complicated_function()){ i = 0; } printf("%d\n",i); } Is this undefined behavior? The compiler can't be sure, because it can't know whether `some_big_complicated_function()` returns true or not. Therefore, it would need to be a runtime check, rather than a compile time check. But runtime checks are expensive, and so it typically isn't done. It would be entirely allowed by the standard to give such a warning at runtime, but the resulting program would go much, much slower due to all the checks.
- sz4kerto 11y ago
  This would certainly worth a warning (at some warning level) exactly because what you say -- the compiler is not sure that the behavior is fully defined.
  - MereInterest 11y ago
    I wasn't sure whether or not it would print out a warning, so I tested it. With g++ 4.8.4, there is no warning printed, even with `-Wall -Wextra -pedantic`.

deleted 11y ago

[deleted]

throwaway999888 11y ago

Clearly what you want is something other than C.

tormeh 11y ago
Ada, and recently Rust, for example. People trying to improve C are sad to watch. People use C because they have an irrational belief in their own faultlessness. Same reason people are afraid of flying but not driving. Selling a correctness-enhancing tool to these people is an impossible task.
- nraynaud 11y ago
  I recently tried to use Ada, that ended in sadness. I couldn't try Rust, because it doesn't (didn't?) run on arm-none-eabi.
  - steveklabnik 11y ago
    Rust should work on `arm-none-eabi` to my knowledge. We don't provide packages out of the box, but people have done it. You need to set up the right target specification, and then it should work.
    - nraynaud 11y ago
      Possible, I don't remember the details. It might also be that setting up a cross compiling toolchain by hand was not exactly how I intended my first contact with a new language be (I had set up a cross compiling GCC, then a cross compiling Ada that ended up in a linux simulator, I might have lazied on the rust test).
      - steveklabnik 11y ago
        Yeah, it's not ideal. We are actively working on making it easier to do these kinds of things; it's not well documented, to be honest.
- 11y ago

imron 11y ago

Surely this is where things like __attribute__ ((optnone)) and #pragma optimize("", off) come in useful.

theseoafs 11y ago

The compiler for this language will be too slow to be useful. Contrary to this author's belief, crypto people really do care about performance. Anyway, what the author wants isn't a "boring C" compiler, it's a compiler for a new language that doesn't actually have undefined behavior.

geertj 11y ago
You realize the author is Dan Bernstein, right?
guelo 11y ago
From the article: > * Claim that a boring C compiler can't possibly support the desired system _performance_. Even if this were true (which I very much doubt), why would it be more important than system _correctness_?
- theseoafs 11y ago
  I read the article. I don't know if the author has considered the implications of what he wrote. What we're talking about is: * Treating all memory accesses as if they were defined * No dead code elimination * Treating all pointers as if they're volatile This adds up to a lot of overhead.
- 0xcde4c3db 11y ago
  It can't be taken for granted that performance and correctness are separate concerns. Even leaving aside the obvious category of realtime applications, performance issues can create DoS vulnerabilities. edit: That being said, a boringcc might end up being less susceptible to these vulnerabilities in practice, since the performance of the emitted code might be more predictable. I'm sure djb has considered this. But in general I think it's often a mistake to think of performance as an implementation detail rather than as a deliberate feature.

tmalsburg2 11y ago

It sounds as if the root of the problem is the C standard. Wouldn't it be better to fix that instead of creating a shadow standard defined by the implementation of this boring compiler? I also object to the presupposition that there is secure software and non-secure software. In most practical cases, it is impossibly hard to tell in which class an application falls, so it would be better not to make that distinction at all.

beefhash 11y ago
>It sounds as if the root of the problem is the C standard. Wouldn't it be better to fix that Isn't the reason for a lot of undefined behavior either in the name of performance through optimization and assumptions, portability because the implementation can just do whatever the underlying platform natively does or the committee just couldn't agree on it? All three of those scenarios appear inherently unfixable to me without a separate sub-standard that intentionally clamps down on performance/ease of portability and actually gets the committee to agree.
- tmalsburg2 11y ago
  I don't know the reasons for why so many things are undefined in C. However, if a program does one thing when compiled with compiler A and another thing when compiled with compiler B, how does that help portability? To my perhaps naive mind this makes portability harder to achieve not easier.
  - pjc50 11y ago
    A classic example of trying to define behaviour regardless of what all the different hardware does is Java choosing IEEE754 floating point. Which isn't natively supported on x86. https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf https://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf If C had standardised to the idiosyncracies of the PDP-11, it might have held back its development.
    - jey 11y ago

tacostakohashi 11y ago

I've always thought that there would be more value in exactly the opposite - a pathological C compiler (or C++ compiler, POSIX operating system, etc.). The pathological compiler would always implement undefined behavior in the most surprising, unpredictable manner - e.g., uninitialized memory would have a properly randomized value that changes from one run to the next, overflows would result in random values, etc. That way, any programs that are made to work on the pathological implementation are largely guaranteed to rely only on standardized behavior, and work on any other standards compliant implementation.

1ris 11y ago
But why? That's a way more complicated and a way less usefull.
- MereInterest 11y ago
  Because if you have undefined behavior, you want your program to crash and burn as quickly as possible, so that you can remove the undefined behavior. What seems to be proposed by Dan Bernstein is not so much a "boring C compiler", but rather a superset of C which does not have any undefined or unspecified behavior.
  - kazinator 11y ago
    Sometimes if you have undefined behavior, you want the documented extension to happen. Or you want the de facto widely understood behavior to happen which has happened on every compiler you've used in 30 years: at least every compiler that was for two's complement matchines. Or every compiler on which pointers to different data types were of the same size. And so on. You also don't want to be burned by code that is relying on a common extension, when that is ported. There is a word for the compiler approach of "we're going to optimize this based on the assumption that the program is not relying on a common extension that is technically UB". That word is: malpractice. For instance, it is not unusual for C code to be targetting machines in which all objects are in the same address space, such that pointers are internally like binary numbers, allowing pointers to different objects to be compared for inequality: ptr1 < ptr2 /* does ptr1 point to a lower address than ptr2 in the one big address space? / Programmers expect this work consistently with the address structure of the machine. They don't want the expression to be somehow wrongly optimized based on the assumption that ptr1 and ptr2 must point to the same object. Implementations must honor requirements beyond those of ISO C. Such as, for instance, behaviors that those same compilers* used to define historically in their past revisions. If GCC has behaved a certain way for 25 years, and some code has come to depend on that, and then that is suddenly taken away, such that entire GNU/Linux distros continue to compile, but break in random places because of that detail, the fault for the regression lies 105% with the doofus who made the GCC commit. Even if that behavior is not defined by ISO C.

pool 11y ago

How far are we from having the will and ability for it to be common to use safer languages than C/C++/Java/etc.? A long time ago, it didn't seem pragmatic to write in, say, ocaml, because no one was going to feel like building or even installing the compiler in order to try your dumb software, but now that everything is about packages from distributions... Today it feels strange to apply security updates for necessary software where the announcements list "Multiple memory safety errors, integer overflows, use-after-frees..."

jbandela1 11y ago

This is actually something the ISO C++ committee is looking at. See https://isocpp.org/std/the-committee https://isocpp.org/std/the-committee - SG12 An example of this is P0145R0 which specifies the previously unspecified order of evaluation. See https://isocpp.org/blog/2015/11/kona-standards-meeting-trip-report https://isocpp.org/blog/2015/11/kona-standards-meeting-trip-... Apparently, the performance difference is expected to be zero on currently popular architectures and around 1% on older architectures.

makecheck 11y ago

One of the bases for security is the age of a program. All other things being equal, the implementation that has been around the longest is expected to have fewer issues than the shiny, new thing (because the old program is well-understood, has been patched to death, considers a thousand corner cases, ran on more hardware, more time passed without any issues, etc.). As such, changing away from well-known compilers and language behaviors may not be the wisest thing for security. On the other hand, extending existing compilers to restrict behavior seems reasonable. Also, by all means document the hell out of the things that are not well-defined.

AndyMcConachie 11y ago

Programmers always want new and more clever constructs in their code, and there is always the next feature that will optimize inner loops 10% better. This is human nature apparently, so don't put too much faith in a boring anything gaining traction. And for those of you courting Rust, remember there are two kinds of programming languages in the world: those that no one uses, and those that are known to be terrible.

eliteraspberrie 11y ago

It exists. It's called CompCert. https://github.com/AbsInt/CompCert https://github.com/AbsInt/CompCert It's free for non-commercial use. I've used it for several months now to build things like Tor. I haven't noticed any disadvantage compared to GCC or Clang.

bpicolo 11y ago
> the generated assembly code is formally guaranteed to behave as prescribed by the semantics of the source C code that suggests to me you could still hit C's undefined behaviors. It's only guaranteeing the semantic equivalence between source and output
ufo 11y ago
I don't think compcert avoids undefined behavior pitfalls. Its more of a way to protect yourself from compiler bugs.
- scott_s 11y ago
  Agreed - I read through their website, and I saw no mention of eliminating undefined behavior.
- nickpsecurity 11y ago
  It's a verified subset of the C standard rather than all behaviors of all compilers. There's some semantics and work covering a lot of that already that just needs to be integrated into CompCert or static analyzers. Likewise, there's static analyzers that already cover it too but not formally verified. Here's an example of investigating that with Compcert: http://robbertkrebbers.nl/research/articles/compcert_formalin.pdf http://robbertkrebbers.nl/research/articles/compcert_formali... Here's a semantics that looks for flaws and undefined behavior: https://www.ideals.illinois.edu/bitstream/handle/2142/34297/Ellison_Charles.pdf?sequence=1 https://www.ideals.illinois.edu/bitstream/handle/2142/34297/...
nickpsecurity 11y ago
You beat me to it. Unlike the other commenters, I see that you're describing it as a robust, predictable start to what Bernstein describes. Undefined behavior and other problems can be caught by extensions to the compiler or via static analysis tools like an Astree Analyzer knockoff. The fact that they'll be written in Ocaml (or SML) will make them more reliable. A good choice and what I've promoted here so far. Additionally, the clean passes are easier to bootstrap for those that fear subversion of their compiler. Can use whatever you want on whatever machine to implement it. So long as code matches spec, the output should be the same. That implementation compiles the other compilers, verifiers, whatever. You might also enjoy that people have already extended CompCert work and others for developments that push in the Boring, Safe/Secure, C compiler direction. Here's some: CakeML - Compile CompCert, etc with this https://cakeml.org/ https://cakeml.org/ Verasco - verified, CompCert-based, static analyzer for C http://compcert.inria.fr/verasco/ http://compcert.inria.fr/verasco/ Verification of LLVM transformations later work (eg Softbound or Code-pointer Integrity clone) http://repository.upenn.edu/dissertations/AAI3592852/ http://repository.upenn.edu/dissertations/AAI3592852/ Extension of CompCert for verified, WCET analysis https://hal.inria.fr/hal-01087194 https://hal.inria.fr/hal-01087194 So, lots of good stuff being done by a subset that focus on right design methods with the right tools. And they're getting incredible results. Surprise! :)

deleted 11y ago

[deleted]

kazinator 11y ago

Some small steps could have been made years ago. For instance, getting rid of unspecified order of evaluation. Or defining a dialect of ISO C in which evaluation is strictly left to right, and making support for that optional (so then your local GCC or whatever can have -fstrict-iso-eval to turn it on). Another example of "small stuff": catching negative values being passed into the <ctype.h> functions. Also, all the instances in the standard library where a null pointer invokes undefined behavior could be required to abort the program. Some undefined behaviors cannot be made defined without a lot of overhead (like catching all out of bounds pointers). However, there is a lot of "low hanging fruit" undefined behavior where things can be improved.

petke 11y ago

> I'd like to see a free C compiler that clearly defines, and permanently commits to, carefully designed semantics for everything that's labelled "undefined" or "unspecified" or "implementation-defined" in the C "standard". That wouldn't just be a new compiler. That would be a new language. I dont think there are any short cuts to fix C. C++ is doing something somewhat similar with the "C++ Core Guideline" rules, which are designed to be checkable by static analysis tool. The long term goal is that eventually C++ will become a safe subset, when all the error-prone parts of the language have replacements. > "Following the rules will lead to code that is statically type safe, has no resource leaks, and catches many more programming logic errors than is common in code today. And it will run fast - you can afford to do things right." https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...

geocar 11y ago
> That would be a new language I don't see why it wouldn't be a valid implementation of C by simply defining things that C doesn't fully define. There's nothing about the definition of "undefined behaviour" that prevents you from defining it in your implementation.
- steveklabnik 11y ago
  Because then you are relying on a specification that's not in the spec. A different interpretation means that your code no longer works. Therefore, you've effectively created a new language.
  - geocar 11y ago
    You argue if GCC and MSVC document their current undefined behaviour, they are creating new languages?
    - steveklabnik 11y ago
      Not if they document it. But if you write a program that relies on those behaviors, you don't have C anymore, you have "GCC-C" or "MSVC-C". I'm also not saying that's inherently a bad thing.

xorblurb 11y ago

At this point you pretty much want an alternative language more than a compiler: one that looks like C, but is completely defined. And this makes kinda sense, yet is IMO still quite dangerous (less than the actual situation but still too much) 1/ There are tons of people who will say: "this project is written in C" instead of the reality that is that this project is written in boringC, (actually probably even most of the projects authors will) and will for ex. compile with a regular C compiler instead of a boringC one, maybe for performance improvement... 2/ You just can't take C, even with a completely defined semantics, and call it a day for secure programming. It would still be filled with crazy features that are intrinsically impossible to render secure (everything about C "arrays" comes to mind) So let's take this idea to its real destination and rewrite secure projects using a really secure programming language, instead of just extending the current mess for 30 more years.

Gracana 11y ago
> At this point you pretty much want an alternative language more than a compiler: one that looks like C, but is completely defined. A "Friendly C" dialect has been proposed: http://blog.regehr.org/archives/1180 http://blog.regehr.org/archives/1180
aikah 11y ago
https://cyclone.thelanguage.org/ https://cyclone.thelanguage.org/ ? kind of ignored by the mass and unsupported now, yet this is exactly the kind of project the developer community needs... a language that is minimalist, yet with enough feature to allow some expressiveness and speed.
- dikaiosune 11y ago
  "(Several of Cyclone's ideas have made their way into Rust.)"
Terr_ 11y ago
So... Java? :P
fbbbbb 11y ago

dikaiosune 11y ago

I'm someone currently in the process of drinking the Rust kool-aid, in part because the goals of the language are very closely aligned with what Bernstein proposes here. Further, because of the memory model, the compiler can produce quite competitive performance while still preventing the execution of any undefined behavior (related to his last strawman in the bullet list). I get that even if Rust offers a panacea to the issues he mentions (which it may or may not, I'm no expert), there's still an awful lot of C code that would need to be deprecated and rewritten. But surely there'd be value in efforts to get away from a language that has such a loose standard?

wspeirs 11y ago
I wonder how hard it would be to create a C -> Rust transpiler. The output would probably be God-awful, but it could be a start to moving away from C towards Rust.
- steveklabnik 11y ago
  Yes, the problem with doing this is that Rust really imposes certain kinds of design constraints that makes this hard. Like, you COULD, but you'd make a lot of use of unsafe, in a way that's not idiomatic. Part of the problem here is that the program may rely on subtleties that make the translation hard. I've seen experienced C or C++ people come into the Rust IRC and ask why certain things aren't allowed, it's often due their code relying on certain aspects of the looser model. That said, it's very possible to go unsafe first and then refactor. It just might not be easy. Another way of making the transition is to pick a component and only re-write it. This is the approach Firefox is taking, for example.
- mcguire 11y ago
  A guess: very, very difficult. Writing code to satisfy Rust's checking is not much harder than writing C[1], but converting arbitrary C to satisfy Rust is something completely different. [1] An' I say it as shouldn't. C is my natural language.
ethbro 11y ago
It's probably going to be a hard row to hoe to get any code that doesn't always visibly break out of circulation. That said, I think that's why there's huge value in moving to languages where you do break things more visibly. Ultimately it comes down to time, and "It looks like it works but might do something wrong sometime" is a weaker argument to delay release than "It will not compile/run".

raverbashing 11y ago

I don't think this can be done with C. Really - Remove all possibilities of memory aliasing. Only exception might be when reading from devices/network (some kind of boxing, unboxing) - All array accesses are bounds checked. - Checked conversion between int/unsigned int and different sized items. Converting a char into an unsigned int? Be explicit in what you want to do - No pointers "flying around", unbound to types. Have reference-counted objects at least. Allow a way of passing a sub-array without copying/with read only permissions. - Remove "gotchas", like ordering of statements, assignment on comparison (unless it's obviously correct), no "compiler dependent" behaviour, no "undefined behaviour" (unless it's machine specific)

sleepydog 11y ago
You are confusing unsafe behavior with undefined behavior. He is not suggesting that a C compiler should prevent the above mistakes. He's suggesting that it should document what code the compiler will emit in such cases. For example, - reading past the end of an array will always segfault the program immediately. or reading past the end of an array will return (arrayType)0 - signed integer types must overflow according to two's complement. A compiler may not assume that, for instance, (int)x < (int)x + 1 is always true. - uninitialized variables are zeroed out. That is, `int *x;` always produces a null pointer, `char y` always produces '0' - A compiler may not remove any statements where it determines such an optimization would not change the behavior of the program. See http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buffer.html http://www.daemonology.net/blog/2014-09-04-how-to-zero-a-buf... Such a compiler would accept and compile any C program, and the programmer (and any static analysis tools) would have a better idea of the behavior of the output program.
- timmytokyo 11y ago
  > reading past the end of an array will always segfault the program immediately. or reading past the end of an array will return (arrayType)0 How would this work when you pass the array (i.e., the pointer) to a function? In order to do what you describe, every pointer would need to include bounds information. This would break the C ABI, so your code would no longer link with existing C libraries.

rwmj 11y ago

gcc -O0 Would that do?

sanxiyn 11y ago
gcc -O0 does not zero uninitialized variables, which is among things explicitly suggested by OP.

lisper 11y ago

It has always seemed to me that the C community (and to an ever greater extent the C++ community) takes particular pride in making it as difficult as possible to write correct code. It provides bragging rights and job security for those select few who have taken the time to master all of the arcana of the language. Personally, I tend to think that life is too short for that sort of thing. That's why I choose Lisp, at least when it's up to me.

nickpsecurity 11y ago
Surprised being a LISPer you haven't read Gabriel's Worse is Better essays on the topic. Convincingly argue the reason behind the uptake of certain tech is they do just enough of a good job to work and for a lot of people. Then they get extended gradually in direction of what they should've been. C and UNIX were perfect examples of this on crappy hardware of 70's-80's. By 90's, it was legacy effect in play for various reasons. Still is. Throw in organizations competing for money or ideology to get a bunch of implementations of a language that was shit to begin with. So, it was more the opposite of what you claim: too many groups trying to make things convenient for themselves in the short-run. LISPers did something similar with their individual messes getting merged into Common LISP. Those doing it clean and sensible on the right hardware were always the outliers.
- lisper 11y ago
  I have read "Worse is better." The point I was trying to make was different: > Then they get extended gradually in direction of what they should've been. Yes, this is the part I'm disputing. It seems to me that C and C++ are not being extended in "the direction they should have been" but rather in a direction that is beneficial mainly to a small incumbent minority. Everyone agrees that security is important, and yet it is virtually impossible to write secure code in C or C++, not because it would violate the laws of physics or some mathematical theorem, but because both the standards and the implementations are hopelessly brain-damaged.
  - nickpsecurity 11y ago

Arnt 11y ago

I don't understand this. He's not asking for a -wundefined or -ansi -pedantic, he's asking for definitions for everything -ansi -pedantic rejects, and a new compiler. Hard work. Which of those cases are that desirable?

tikums 11y ago

OpenBSD (+LibreSSL) folks have been uncomfortable with unpredictable compiler optimizations for quite a while too. Here's a previous discussion on compilers in OpenBSD: https://news.ycombinator.com/item?id=9322259 https://news.ycombinator.com/item?id=9322259

ape4 11y ago

In the past initializing variables to zero might have been too slow. But not now.

gdwatson 11y ago

I think the biggest problem with undefined behavior in C is that it is taken to justify violation of C's customary semantics[1] on platforms where it isn't necessary. Many types of undefined behavior in the C standard are essentially dispensations for Lisp machines or one's complement CPUs. It's very surprising to see them invoked by the optimizer on more conventional platforms. It might not satisfy djb, but a standard for these behaviors alone would be great progress.[2] [1] http://arcanesentiment.blogspot.com/2014/12/customary-semantics.html http://arcanesentiment.blogspot.com/2014/12/customary-semant... [2] Posix doesn't really address this, sadly. The committee seems to have avoided standardizing byte size until an unforeseen interaction with a later C standard pushed it into a corner.

pcwalton 11y ago

Required reading here is Chris Lattner's "What Every C/C++ Programmer Should Know About Undefined Behavior", which goes into good detail as to why undefined behavior is important for performance, sometimes very important: http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html?m=1 http://blog.llvm.org/2011/05/what-every-c-programmer-should-... Could you make a new language that doesn't depend on UB for performance? Sure, quite possibly. Is C that language? Probably not.

cbd1984 11y ago

He says this: > Claim that a boring C compiler can't possibly support the desired system _performance_. Even if this were true (which I very much doubt), why would it be more important than system _correctness_? If correctness is more important than performance, C is the wrong language. Consider a language with, at the very least, garbage-collection built into the core, as memory allocation and usage errors are the base of a lot of what goes badly wrong with C programs. In short, his notion of "Boring C" seems to be Haskell or Common Lisp.

nickpsecurity 11y ago
Those languages are incredibly complicated. I'd go with Modula-2 or Pascal as a boring C candidate.
- cbd1984 11y ago
  I'm not sure that either of those languages are usually GC'd, but they'd be good choices, too, if they were. (I just don't like Wirth languages. I don't like their type systems and I don't like the definition-before-use requirement. They're better than C, in some respects, but worse in others, and the net isn't always an improvement.)
  - nickpsecurity 11y ago
    All of Wirth's after Modula-2 were GC'd. C programmers would reject a GC'd language, though. So, I stayed with originals that enhanced safety, predictability, readability, and so on while capable of low-level, fast stuff. Wirth's stuff and industrial variants of it often let you do unsafe stuff in dedicated modules for that so you could turn off GC or checks. Plus they wrote OS's in Modula-2 and Pascal on ancient hardware. Whole platforms. So, they're proven for the use case C programmers often say you need their language for.
    - cbd1984 11y ago
      Whole OSes were written in Lisp, too, as well as other GC'd programming languages (Smalltalk, Mesa, perhaps a few others), and they ran on rather underpowered hardware by modern standards as well. I get your point, but having to check for allocation failure is a substantial source of bugs in non-trivial programming without GC, and having to manually allocate RAM makes string handling, for example, orders of magnitude less safe than it is in a language with GC.

cornholio 11y ago

Event the greatest brains can fart, this proposal can be safely ignored.

chris_wot 11y ago

Is there any list of undefined behaviours in C? Other than the standard.

aplorbust 11y ago

Actual URL, accessible without Javascript: https://groups.google.com/forum/message/raw?msg=boring-crypto/48qa1KWignU/o8GGp2K1DAAJ https://groups.google.com/forum/message/raw?msg=boring-crypt...

Too 11y ago

I think the problem is not the language standard per se, it's the education and historically sloppy compilers. It's a bit like W3Schools for javascript teaching everybody all the worst practices, that's not a problem with the javascript language itself. I mean really, what are the most common cases of undefined behavior that cause real problems in the world? (Reading past array bounds or trying to dereference uninitialized variables are such obvious errors that i don't categorize them as 'unexpected undefined behavior', the later is caught by most compilers and static analyzers today anyway. Though i agree having the length known in arrays would greatly help.) Any time i see some raging blog post about how undefined behavior ruined someones day it's 99% of the time because they one way or another were trying to cast a pointer of one type to a pointer of another type, like thinking they could read the first 16 bits of an int* by casting it to a short* first. The root cause of this is because people think that pointers are addresses to a sequential grid of NAND-gates on your DDR4-stick, when they in fact are abstract references to a variable that can carry a value. This comes from school, where the way teachers show you how pointers work is exactly like that. Tutorials on the internet also say that you can do smart tricks with pointers just to teach you and then people think that's how you should do it. When compiling in debug-mode this is how the memory is usually aligned so things still work anyway. Historically compilers didn't warn about this either and things still worked so many people got used to writing things this way and today those people are the "experienced C gurus" in companies that pass down this misbehavior to their younger peers. Overflow on signed arithmetic is another of the common pitfalls but that's life if you code for embedded, the alternative solution that "just works" would be to do all arithmetic in a BigInt type as python but that clearly isn't viable on embedded, or to do everything in floating point but that has its own pitfalls.