5 ms·
I don't think he's making the argument you think he's making. This is clearer in his later comments in the thread, in which he says that UB is much scarier in s
by samth 11y ago
I don't think he's making the argument you think he's making. This is clearer in his later comments in the thread, in which he says that UB is much scarier in systems like Fossil.
His argument is that achieving the level of quality that SQLite has requires verification (in a broad sense) after the compiler, and that's what he does. If you consider the goal to be producing quality-assured binaries, then you can treat UB, compiler bugs, and many other things as falling in a similar category, which are almost certainly eliminated by an MC/DC test suite.
As you say, this isn't feasible for almost any software (as John said in the blog post, SQLite is the only program he knows of that has MC/DC testing when not required by law). But it does mean that rewriting SQLite in Rust wouldn't provide as much value as rewriting many other things where the binaries do not have such guarantees.
- pcwalton 11y ago> His argument is that achieving the level of quality that SQLite has requires verification (in a broad sense) after the compiler, and that's what he does. > If you consider the goal to be producing quality-assured binaries, then you can treat UB, compiler bugs, and many other things as falling in a similar category, which are almost certainly eliminated by an MC/DC test suite. I don't think that they're eliminated because dynamic testing can't eliminate everything—it only finds bugs given its test inputs. Moreover, though, I'm also skeptical of the claim that binaries are all that matter. Lots of software projects (for example, Firefox) import SQLite as source into the project instead of using the binaries. They upgrade their compilers without running the SQLite test suite to catch regressions. (Firefox might, but I'm sure lots of other projects using SQLite from source don't.) > But it does mean that rewriting SQLite in Rust wouldn't provide as much value as rewriting many other things where the binaries do not have such guarantees. Sure; static analysis is more useful in systems that aren't as well dynamically tested. But static analysis still has value.
- samth 11y ago> I don't think that they're eliminated because dynamic testing can't eliminate everything—it only finds bugs given its test inputs. I can't think of an example of UB-exploit in the compiler that wouldn't result in different branch behavior, and thus, I think, in failure of MC/DC. But I may simply be insufficiently imaginative. John makes the point about source code in a follow-up comment, and I agree, but I also see Hipps' perspective.
- lomnakkus 11y ago> I can't think of an example of UB-exploit in the compiler that wouldn't result in different branch behavior, What about Implementation-defined Behavior? This is (I think!) technically a subset of Undefined Behavior and it permits such things as setting values[1] to arbitrary (but well-defined!) values on various operations, such as "excessive" left shifts. What I'm saying is that a compiler is permitted to substitute IB for UB and still be conforming. So it could start to do strange things to arithmetic, etc. Does that make sense as an example of what you're thinking of? EDIT: [1] I obviously meant memory locations... as referred to by "variables" which aren't really variables, but are really binders/aliases. But here we are.
- samth 11y agoI think you're confused about implementation-defined and undefined behavior. The former is not a subset of the latter, but excessive left shifts are UB.