3 ms·
> This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this. "Rust doesn't eliminate all bugs" is a rephrased versi
by pcwalton 11y ago
> This is itself a straw man. Follow the link to Mr. Hipp's comments and read them. He did not say this.
"Rust doesn't eliminate all bugs" is a rephrased version of "Some well-formed rust programs will generate machine code that behaves differently from what the programmer expected."
> That a programmer who has produced such high-quality and rigorously tested software as sqlite should be portrayed as either cavalier or naive about software quality is something I find profoundly mis-guided.
I don't think he's cavalier or naive about software quality! SQLite's quality speaks for itself, and he is completely correct about what you need to do to achieve the level of correctness that SQLite has. He knows a lot more than I do about the rigor needed to ensure that amount of software quality. I do think, though:
1. The amount of testing that SQLite undergoes is not economically feasible for most software.
2. Rust's testing tools would allow for the same level of code quality in the absence of evidence otherwise. (The example he cited, code coverage, is wrong, as kcov is available for Rust.)
3. There is value in static analysis above and beyond the value of testing (also vice versa), because testing only reveals bugs that manifest themselves in inputs available in the test suite. This is true regardless of the code coverage of the tests.
In other words, I think Hipp's argument would make sense if it were something along the lines of: "I couldn't test a Rust SQLite as well as I can test the existing SQLite because kcov is missing features X, Y, and Z, and the value I get from the static analysis is less than the value I get from those code coverage tools, since we've found lots of bugs from those code coverage features and haven't found as many memory safety/data race bugs". That'd be totally valid. But as a blanket statement that Rust would make things worse, it doesn't make sense to me.
- jstimpfle 11y ago> "Rust doesn't eliminate all bugs" is a rephrased version of "Some well-formed rust programs will generate machine code that behaves differently from what the programmer expected." I think that sounded more like "Many of the programs which are UB in C are also UB in Rust, even though not (yet) specified as such."
- pcwalton 11y ago> I think that sounded more like "Many of the programs which are UB in C are also UB in Rust, even though not (yet) specified as such." Well, this seems either (a) false or (b) uninteresting to me. It's false because C and Rust have different semantics, and Rust rules out lots of programs that C doesn't. It's uninteresting because if the point is that Rust has accidental undefined behavior due to compiler bugs (and it does), then that UB is so rarely hit that it doesn't matter nearly as much in practice as the UB in C. CPUs have bugs, too. Do we consider Java an unsafe language because of rowhammer?
- jstimpfle 11y agoHelp me out here, I don't know much about Rust. Which bad programs does Rust rule out? I know about the borrow checker, but I don't think ownership bugs is a type of bug that Mr. Hipp frequently produces. It's a program design issue -- not something you think about at every single line you write. A well-designed program does not do many ownership transfers. As someone else noted, out-of-bounds errors are sadly a pain to debug in C. For testing code, C has at least valgrind (and certainly many less well-known tools). For production code, you might not want dynamic index checking. It would invalidate performance arguments for Rust. I think the real nasty cases of UB in C, which frequently occur as not statically refutable, are signed arithmetic overflow and shifts. As with out-of-bounds errors, I don't think Rust has a better story here -- only better built-in tooling.
- viraptor 11y agoRust allows you to specify the behaviour you want: Checked operations: https://doc.rust-lang.org/std/primitive.i8.html#method.checked_add https://doc.rust-lang.org/std/primitive.i8.html#method.check... Saturating: https://doc.rust-lang.org/std/primitive.i8.html#method.saturating_add https://doc.rust-lang.org/std/primitive.i8.html#method.satur... Wrapping: https://doc.rust-lang.org/std/primitive.i8.html#method.wrapping_add https://doc.rust-lang.org/std/primitive.i8.html#method.wrapp... Wrapping with notification: https://doc.rust-lang.org/std/primitive.i8.html#method.overflowing_add https://doc.rust-lang.org/std/primitive.i8.html#method.overf... These apply to shifts too and you can tag types themselves to always behave one way. Overflows at compile time are caught as errors. More info at: https://github.com/rust-lang/rust/issues/22020 https://github.com/rust-lang/rust/issues/22020 If the requested behaviour is unspecified (just `a+b`), overflows cause runtime panic (edit: or wrapping behaviour in release build) See https://doc.rust-lang.org/1.7.0/reference.html#behavior-not-considered-unsafe https://doc.rust-lang.org/1.7.0/reference.html#behavior-not-...
- samth 11y agoI don't think he's making the argument you think he's making. This is clearer in his later comments in the thread, in which he says that UB is much scarier in systems like Fossil. His argument is that achieving the level of quality that SQLite has requires verification (in a broad sense) after the compiler, and that's what he does. If you consider the goal to be producing quality-assured binaries, then you can treat UB, compiler bugs, and many other things as falling in a similar category, which are almost certainly eliminated by an MC/DC test suite. As you say, this isn't feasible for almost any software (as John said in the blog post, SQLite is the only program he knows of that has MC/DC testing when not required by law). But it does mean that rewriting SQLite in Rust wouldn't provide as much value as rewriting many other things where the binaries do not have such guarantees.
- pcwalton 11y ago> His argument is that achieving the level of quality that SQLite has requires verification (in a broad sense) after the compiler, and that's what he does. > If you consider the goal to be producing quality-assured binaries, then you can treat UB, compiler bugs, and many other things as falling in a similar category, which are almost certainly eliminated by an MC/DC test suite. I don't think that they're eliminated because dynamic testing can't eliminate everything—it only finds bugs given its test inputs. Moreover, though, I'm also skeptical of the claim that binaries are all that matter. Lots of software projects (for example, Firefox) import SQLite as source into the project instead of using the binaries. They upgrade their compilers without running the SQLite test suite to catch regressions. (Firefox might, but I'm sure lots of other projects using SQLite from source don't.) > But it does mean that rewriting SQLite in Rust wouldn't provide as much value as rewriting many other things where the binaries do not have such guarantees. Sure; static analysis is more useful in systems that aren't as well dynamically tested. But static analysis still has value.
- samth 11y ago> I don't think that they're eliminated because dynamic testing can't eliminate everything—it only finds bugs given its test inputs. I can't think of an example of UB-exploit in the compiler that wouldn't result in different branch behavior, and thus, I think, in failure of MC/DC. But I may simply be insufficiently imaginative. John makes the point about source code in a follow-up comment, and I agree, but I also see Hipps' perspective.
- SQLite 11y agoI was not aware of kcov or its basis bcov. Thanks for pointing those out. However, a quick glance at the bcov source code leads me to believe that it only does source-line coverage, not branch coverage. So, unless my quick reading of bcov sources is mistaken, I couldn't test a Rust SQLite as well as I can test the existing SQLite because kcov/bcov is missing the ability to measure coverage of individual machine-code branch instructions, and the value I get from static analysis is much less than the value I get from branch-coverage testing tools. That's not being pedantic, btw. The difference between source-line coverage and machine-code branch coverage is huge. The latter really is necessary.
- pcwalton 11y agoThanks, that's really great feedback. We should invest in making use of llvm-cov. It should dovetail nicely with the MIR work (which is now to the point where it can build the Rust compiler), since the language-specific IR it exposes is especially suited for quick development of instrumentation passes.
- caf 11y ago1. The amount of testing that SQLite undergoes is not economically feasible for most software. But the argument isn't about "most software" - it's about SQLite in specific. And that's the crux of the issue. Saying that he believes it would be counterproductive to rewrite SQLite in Rust at this time is not saying that that would be true of all or most or some other programs.