3 ms·
> As of early 2011, the under-development version of CompCert is the only compiler we have tested for which Csmith cannot find wrong-code errors. This is not fo
by carapace 7y ago
> As of early 2011, the under-development version of CompCert is the only compiler we have tested for which Csmith cannot find wrong-code errors. This is not for lack of trying: we have devoted about six CPU-years to the task.
Buries the lede?
Sure formal methods have a high overhead but for compilers maybe it's worth it?
- nullc 7y agoI felt like that was kind of misleading when I read that section. Compcert is hardly an optimizing compiler compared to GCC. And bugs were found in it (just not middle end). It's unclear to me how the defect rate really compared for equivalent performance code (which is probably -O0 or maybe -O1 gcc). CompCert also had correctness as a headline goal in a way that other compilers have not, so because of these differences it's hard to tell how much impact formal methods were themselves have. Though it its reason to be hopeful at least.
- ncmncm 7y agoCompCert does not generate production-grade code. Which is to say, it does not optimize as well as production-grade compilers. There are circumstances where this is not so important; anywhere one would otherwise use a slow language, like Javascript or Java, CompCert output might be fast enough. More important, though, is that CompCert output is not, generally, more correct than its input. So unless you are formally verifying the program source, CompCert is unlikely to make your resulting system more correct. Most of us go years without finding a bug caused by our compiler. Furthermore, unless you are formally verifying your system specification, a formally-verified program may not help much. Being definitively wrong is rarely much better than accidentally wrong. But what do you use to verify your specification? Formal methods can verify numerous properties, but correctness is not among them. Similarly: programming language standards -- specifications -- are also not formally verified. Correctly implementing a buggy language design gets you little closer to a correct system. Testing finds bugs in all layers, but not reliably. Bugs found in testing tend to be implementation bugs, at first, but soon testing finds mainly specification bugs.
- carapace 7y agoI'm hearing, "The tolerances on our gears aren't that great to begin with so why care if sand is falling in?" You're making good points, but the undeniable fact that CompCert is not a panacea doesn't contradict the fact that other compilers are buggier?
- ncmncm 7y agoOther compilers are buggier in ways that demonstrably does not much affect the correctness of systems built with them. It's a matter of managing expectations. How much less-buggy would systems be, if built with compilers that are less buggy? The number is probably not negative, but the space between it and zero would be hard to see. So it is an interesting achievement, academically, but a long way from industrially useful. Another way to say it is that it is a necessary step in achieving verifiable systems, but there are many other such steps yet to be taken.
- carapace 7y agoOh, well, if you're going to be reasonable about it... :) I still feel like the conclusion is wrong, so one of the axioms or deductions must be wrong, but I can't quite put my finger on it. In any event, well met.