11 ms·
I expect it's because because the first part is essentially preaching to the choir, and because Richard Hipp very much disagrees with the second part[0] > Rewr
by masklinn 11y ago
I expect it's because because the first part is essentially preaching to the choir, and because Richard Hipp very much disagrees with the second part[0]
> Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt.
(see link for expansion on that matter, which is a question of tooling and testing)
[0] http://blog.regehr.org/archives/1292#comment-18452 http://blog.regehr.org/archives/1292#comment-18452
- kbenson 11y agoProf. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate whatever machine code it wants without it being called a compiler bug. That’s an important finding. But as it happens, no modern compilers that we know of actually interpret any of the SQLite source code in an unexpected or harmful way. At some point some popular compiler is going to make a subtle but important change to some undefined behavior that's not going to be immediately obvious as to it's repercussions, and the fallout will be massive. It boggles my mind the mental contortions people will go though to justify what is essentially an argument of "it hasn't caused a problem yet" while ignoring that it's caused many problems already, just not that they've noticed or that have affected them.
- catawbasam 11y ago"while ignoring that it's caused many problems already" Really? I believe Mr. Hipp is claiming that it hasn't. Do you have evidence to the contrary? I don't think the argument is that these cases should be ignored -- they are now corrected, after all. It is that most of these cases should be treated as low priority compared to issues that are creating observable problems.
- Jtsummers 11y agoI take kbenson to be saying that what Mr. Hipp claims isn't a problem for SQLite is or has been a problem for other projects. That is, other projects have been bitten by relying on undefined behavior (such as what a variable's initial value might be) that may have been consistent across a variety of implementations, and then suddenly wasn't. Perhaps it's safe to say that today we don't need to address these issues on this project in this language, but that's taking on technical debt and setting ourselves up for work in the future (that may be difficult to identify at that time).
- kbenson 11y agoThat was exactly my point, and I think the alternative interpretation makes no sense in light of the portion of my comment saying "just not that they've noticed or that have affected them." > Perhaps it's safe to say that today we don't need to address these issues on this project in this language, but that's taking on technical debt and setting ourselves up for work in the future (that may be difficult to identify at that time). Yes, that is succinctly saying something I was just implying. Even if a current C program is verified as having absolutely no problems with any current compiler due to the way it is using undefined behavior and the compilers interpret it, it's impossible to assume it will remain in that state by the nature of the problem being examined. Undefined behavior is undefined, and thus may change. Now, any language may decide to change how something works, so every program has to deal with this at some level, but again by the nature of this problem, there's much less assurance that the problem won't be a small,subtle change, possibly in one compiler, which is missed until it's widespread. This would be much less of an issue if there was a specific subset of C which defined most the undefined behavior which could be turned on with a flag. It would probably prevent portability in some respect, but I hope we've finally reached a place where portability is accepted as secondary to security.
- kbenson 11y agoI was making a general point using his statement as the impetus, not a specific point in the case of SQLite. Undefined behavior (specifically, developers relying on it being consistent) has caused many problems in the past, and will undoubtedly do so in the future. Relying on behavior which by it's definition is impossible to rely on, is not a situation I think we should be defending.
- qznc 11y ago> The disagreement is not over whether or not [undefined behavior] is a problem, but rather how serious of a problem. Is it like “Emergency patch – update immediately!” or more like “We fixed a compiler warning” or is it something in between. –Hipp In this specific case with sqlite it certainly is more of a fixed-compiler-warning. However, you could imagine some undefined behavior in SSL implementation to require emergency-patch level as well. I agree with you that undefined behavior "in most cases should be treated as low priority".
- Dylan16807 11y agoIt's still important to distinguish the two. The source code has bugs, but in the year 2016 it builds binaries that do not have bugs.
- cbsmith 11y agoReally, if your project intends to be portable C code, you need to compile it with undefined behaviour generating an error.
- adrusi 11y agoUndefined behavior is not always identifiable through static analysis. Obviously it can be checked against at runtime, but that's actually quite expensive. It would, for example, include bounds checks for everything, and overflow checks on all signed arithmetic.
- pcwalton 11y agoThat's not the worst of it: the truly intractable part is preventing use-after-free UB. The only ways to do this are (a) remove malloc from your language; (b) add a lifetime system (incompatible with all existing C libraries); (c) add a garbage collector (which most projects written in C will not accept for performance reasons).
- astrange 11y agoAnd once you've eliminated that, you have DoS bugs like forcing infinite loops, abandoned (referenced but unused) memory leaks, and worst-case hash table insertions. All of those are serious attacks for anything with a resource budget.
- cbsmith 11y agorlimit ftw
- kbenson 11y agoI think that's mostly solved by changing the whole of the idea from "remove/disallow all undefined behavior" to "remove as much of the common needed undefined behavior as possible such that most programs need not really use it". Perfect is the enemy of good.
- ktRolster 11y agoAt some point some popular compiler is going to make a subtle but important change to some undefined behavior that's not going to be immediately obvious as to it's repercussions, and the fallout will be massive. Wait until you realize that the length of a byte in C is not clearly defined. Someday a processor will come along where a byte is 6 bits, and the fallout will be massive. (Really, it's happened before). No, actually that processor would not become popular because on one would use it. The reality is unless you are using a formally-defined language like ML, you are relying on undefined behavior in your language.
- tatterdemalion 11y agoIts far more likely for a major compiler to exploit more undefined behavior for optimizations than for a processor with a bizarre sized byte to become popular. The major compilers already do this. > The reality is unless you are using a formally-defined language like ML, you are relying on undefined behavior in your language. This is not true, not using the definition of "undefined behavior" provided in the C standard.
- _kst_ 11y agoThe number of bits in a byte is specified by the CHAR_BIT macro, which is required to be at least 8. It's commonly larger than 8 on C compilers for some DSPs, but almost universally 8 for hosted implementations. POSIX specifically requires it to be exactly 8.
- jerf 11y agoDid you see the short-lived attempt to create "friendly C" a few months ago? [1] There's languages that have a few undefined corners, and then there's C. A sufficiently large difference in quantity becomes a difference in quality. C is qualitatively worse than most modern languages with its undefined behavior. (Granted, some modern languages escape by having the one implementation, which is then the definition. But still, that's less undefined than C.) [1]: http://blog.regehr.org/archives/1287 http://blog.regehr.org/archives/1287
- Dylan16807 11y ago
- TheCondor 11y agoCan you suggest what that change might be related too? I, too, think it's less likely than one might think. This discussion seems to miss a couple things. SQLite is embedded quite often, by programs written in C; would embedding a rust library and possibly runtime fix things? The parent program would still potentially have defects and those defects could impact SQLite. SQLite is also rather mature, I do t see how you can compare a rewrite to nature code. Take OpenSSL as an example, they are t tossing it, they are fixing it, it's a much more shallow lift to fix it. I'm all for some big rust programs to prove its case though, a mailer, a dns daemon, some sort of database. Something useful cut from whole cloth, and ideally something we have historically not done well. I don't think a rewrite is it though.
- kbenson 11y ago> Can you suggest what that change might be related too? I, too, think it's less likely than one might think. Aliasing rules[1]? Compilers apparently don't agree on that now, so even if it were to coalesce into the same undefined behavior, one would have to change. > This discussion seems to miss a couple things. SQLite is embedded quite often, by programs written in C; would embedding a rust library and possibly runtime fix things? I'm not advocating for Rust as much as I'm advocating against C, or at least against C as it currently exists and is implemented with so much undefined behavior. I've argued elsewhere in this discussion that a special subset of C with as much undefined behavior as possible specifically defined, which could be enabled through a flag, would do wonders (even if at the expense of some portability). > Take OpenSSL as an example, they are t tossing it, they are fixing it, it's a much more shallow lift to fix it. It's much more shallow to review and patch it. Let's not kid ourselves that it will be fixed when they are done with it. There will likely be bugs regardless of the language used to implement a crypto library. Does that mean we should ignore when one language allows an entire class of bugs that another does not, especially when it's for a Crypto library? Let me make my case another way. What if C nevercaught on as the dominant language, or C was defined originally in a more strict manner, and most the utilities and tools we take for granted were instead implemented in a language that had less undefined portions, allowed less undefined behavior, and thus had less bugs and security problems? Would the resultant small performance hit (due to being unable to optimize around the undefined behavior) outweigh the added stability and security, or would we have been better off? I think we would have been better off, and I think since C's still widely in use, it's not too late to make that case. 1: https://news.ycombinator.com/item?id=11288665 https://news.ycombinator.com/item?id=11288665
- haberman 11y agoHe certainly has a point, but I think he goes too far in assurances about the current sqlite3.so. Testing "every single instruction" does not guarantee that all C-level UB has been eliminated, because some bugs can be input-dependent. For example, even if your coverage tells you that this function has been tested, it could still trigger undefined behavior for other inputs that trigger overflow: int f(int x) { return x * 2; }
- pcwalton 11y agoFor another example of this issue, consider the fact that data race detectors do not eliminate all data races in practice, because they only find races that show up on your test inputs.
- paulmlewis 11y agoI ask this from a position of seeking knowledge rather than being adversarial, the discussions on undefined behaviour from a C language perspective always seem muddled to me between what's undefined at the C language level such that a compiler might take advantage of it and produce results that would be unexpected looking at the code, vs. the machine code generated faithfully represents the language statements but a bug may be present when processing some external input at runtime. If the inputs here are coming from something external to the program, such that the compiler can't know the value of x then the machine code should be faithful to the language statements, it's a bug if overflow occurs which may have other runtime implications, but the compiler isn't going to remove some chunk of code, etc. because of it. From their testing page SQLite say they also test boundary conditions (I don't know to what granularity that is though) which which may catch something like this, which is agreeing that instruction level coverage isn't enough. They also run their tests with all the various sanitisers enabled. Isn't this the same case in Rust, the non-release builds would need to see a suitable test case for the overflow check to cause a panic. I guess I'm asking is this example relevant to concerns about undefined behaviour and optimising compilers vs. Rust? Since the function is input value dependent in practice isn't this more like implementation defined behaviour at runtime, in that a platform will alway provide a consistent behaviour, e.g. overflow, trap, saturate, etc.? I've followed regehr's blog for a few years, I've read a lot about Rust, but I'm mostly working in higher level dynamic languages and don't have a lot of hands on experience with C or Rust and just wondering if I'm missing some subtlety here.
- pcwalton 11y agoDisappointing to see Hipp make that argument. It's trivially refuted. Yes, all programming languages allow the programmer to write bugs. But languages very much vary in how many, and what kinds of bugs programmers write in practice. Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man. If you want to argue that Rust isn't worth it, you need to convince me that C plus gcov results in fewer bugs in the important areas in practice than Rust (plus kcov [1] if you like) does. I think that's going to be pretty hard. (Especially if memory safety issues are the most important bug class you're concerned about: I think it's completely impossible for any C-based solution to compete with Rust here, regardless of how much tooling you add.) Drawing an equivalence between undefined behavior and compiler bugs also doesn't make sense. Compiler bugs are way way less commonly encountered than undefined behavior in C. Also, they're qualitatively different: compiler bugs get fixed in new compiler versions, while UB is by design and doesn't get fixed. [1]: https://users.rust-lang.org/t/tutorial-how-to-collect-test-coverages-for-rust-project/650 https://users.rust-lang.org/t/tutorial-how-to-collect-test-c...
- kbenson 11y ago> Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man. Yes, it's very much akin to saying "well, a seat belt won't eliminate all deaths". I would understand an argument of "I'm not prepared to rewrite the software at this time" or "I'm not familiar enough with replacement X to assess whether it's a good choice", but at some point more people will have to acknowledge C's failings with more than lip-service. That doesn't mean Rust has to be the solution/replacement, but something does.
- catawbasam 11y ago'Saying "well, Rust doesn't eliminate all bugs" is attacking a straw man.' This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier or naive about software quality is something I find profoundly mis-guided.
- pcwalton 11y ago
- jfoutz 11y agoThe quality of tooling, and the ability of experts to verify the output of machine code is a really important point. I think I'd agree, rust at this point would hold back an elite developer like Richard Hipp. The promise of rust, which may or may not be realized is pushing some very common problems down to the compiler. All code has bugs, so the compiler probably does things wrong in some cases. As the tools mature, these will get scrubbed out, just like every other project. That said, array bounds checking is responsible for so many problems, it seems worth it to raise the minimum for a language. Not every developer is elite. In fact, they're pretty rare. C requires you to be really smart all the time, or at least be aware of when you're not smart enough to get a chunk of code right. Looping over some bytes from a file shouldn't be that risky. rust lets me, a less than elite developer, save my few moments of brilliance for the hard part of a program, rather than having to worry about the evaluation order of foo(i++,i++); Maybe, it'll turn out the only way to make good software in the future is to find the best 100 developers in the world, and get them to make stuff. I doubt it though, being able to leverage the other million of us to make stuff, and have some (a lot?) of confidence it'll be free from the most common C errors is valuable. Nobody should be forced to use tools they don't like. rust is trendy, but it has some very good ideas. Trendyness isn't reason alone to dismiss its approach. blah blah rust cheerleading blah blah.
- pcwalton 11y agoI think the idea that "elite" developers can write bug-free C (or even just network-facing C free of security-sensitive memory safety problems) is pretty well refuted at this point. Now you can write bug-free C if you're willing to spend enormous time and money on testing: this is to a first approximation what SQLite did. But that only makes economic sense for a small minority of projects. Just putting "elite" developers on a C project, without the huge verification cost, is by itself not enough to eliminate bugs, as much as we as hackers would like to think it is. (The one exception may be, like, DJB. But djbdns/qmail are very unusual C programs in many ways.)
- reynoldsbd 11y agoIt will be interesting to see if the formal proof/verification work that is being done by NICTA for the seL4 project will mature into something that can be used practically elsewhere in industry. https://sel4.systems/ https://sel4.systems/
- nickpsecurity 11y agoWow. I was surprised that comment came from a smart, accomplished guy. It's nonsense. What he's essentially saying that the existence of compiler bugs in Rust refutes using it as a C alternative. Let me try that: the existence of hundreds of compiler bugs in C compilers over past years means it can be relied on either. CompCert had a few at spec level so we throw it out, too. Realistically, you rewrite the components piece by piece in the new language. You make sure each compiles right with testing and review. You report any problems to the compiler team, who fixes them. Eventually, you're whole app is in the safer language without a lot of work. You might even swap them where safer one becomes reference code with other one there for any platforms not supported yet or too buggy. A diversity benefit as Hipp mentions. With all of this, the program becomes immune to most memory & concurrency issues while being easier to maintain. Its undefined behavior will probably be a fraction of C's in number and severity. That's a net win. Note: I'd have told him Ada/SPARK instead of Rust given it's been stomping C in embedded safety for a long time w/ lots of tooling for verification activities already there. Counters his compiler maturity argument, too.
- teraflop 11y ago> What he's essentially saying that the existence of compiler bugs in Rust refutes using it as a C alternative. That is not at all what Hipp's comment says, and the original author's response to it gives it a much more charitable reading than you seem to be.
- nickpsecurity 11y agoLet me re-read it. He makes several points. The first is that they're intentionally relying on undefined behavior known to cause problems sometimes out of nowhere. He says it's OK they rely on it because it's not currently causing them problems. Relying on something impossible to rely on as kbenson worded it since that's working out so far. Reminds me of a George Carlin quip about people building villages on active volcanos then surprised when lava turns up in living room. Pretty foolish. Next he points out Rust should reduce undefined behavior occurrences vs C while eventually having some of his own. That's correct but he neglects the biggest benefit: its safety scheme preventing many flaws found in C projects by default. Leaving this out of his comment makes it Rust vs C on undefined behavior and compiler correctness only. Bad comparison given efficient, memory safety is basically Rust's main benefit. Next, he conflates compiler bugs with undefined behavior. They're not equivalent. One is an implementation failure to be remedied. One is a design failure to probably stay indefinitely in the language and compilers. He's falsely reframing the situation to prop up an argument. Next, he delivers the argument: that compiler bugs mean you can't rely on Rust unless you check the machine code itself. I like that he checks machine code as that's a high-assurance recommendation with proven value. He claims there's not enough tools to get the job done and a lack of compiler diversity. The first might be true and the second usually only matters if they're implementing the same spec. Otherwise, you're getting effectively different programs you can't compare directly. Plus, most GCC, LLVM, and Rust programs are performing just fine relying on one, actively-developed compiler without machine code testing. So, he's made some bogus claims, dismissed Rust's whole benefit package, focused discussion on machine code from buggy compilers, made claims about its verification which I lack knowledge to evaluate, and ignored field evidence that his focus area is a small problem. He's trying really hard to dismiss Rust entirely at compiler level without much to show for it. Incidentally, it takes much less writing for most of us to dismiss C on grounds of language or compiler safety. Something you don't see him doing. ;) That said, Rust community should invest in tooling for assembly/machine level verification if he was correct in saying they don't have it. That will be important for OS and embedded where developers trust compilers very little. Past that, his comment's misdirection and level of bias deserves no charitable interpretations.
- outworlder 11y agoFrankly, SQLite is the exception that proves the rule. For all the complexity of the SQL Language, and efficiency constraints SQLite needs to have, plus all the algorithms it has to implement, it still got a pretty well-defined task. It is not easy to transfer that experience to other systems. Say, your company's backend API. Completely different constraints. It could still be the case that, had Rust existed when SQLite was being created, that it would have taken much less engineering effort. Which is the metric that matters, as given infinite manpower, you can write anything in any language. I do agree that rewriting SQLite now, which is a very battle-tested piece of software, would probably do more harm than good. I bet people will still try, for fun if nothing else.