4 ms·
Yes, a full test suite. 100% code coverage is a thing, even in compiler validation. Also, every test that was ever written for a compiler is still out there,
by fit2rule 6y ago
Yes, a full test suite. 100% code coverage is a thing, even in compiler validation.
Also, every test that was ever written for a compiler is still out there, being run, somewhere.
And, for the record, the real world is pretty big.
- andrewaylett 6y agoBut 100% code coverage isn't necessarily a useful thing in this context, as different code (and different options) mean different interactions between passes. And you're never going to cover all the different paths through the compiler. Back when I worked on a commercial compiler, we had a smoke test which we ran before committing. It would take ~30 minutes to run. We had a longer, roughly eight hour, set of tests which we ran daily. And we had machines running tests 24/7, which managed to clock up roughly 20 million distinct test cases (combinations of code and options) over the course of a year. Every single compiler bug we'd ever seen was present (in a reduced form) in the conformance tests we'd run before release. That took about a week.
- fit2rule 6y ago>And you're never going to cover all the different paths through the compiler. Depends who you work for and how willing they are to let such justifications stand in a court of law.
- joosters 6y ago100% code coverage does not mean a full test suite. You can still find huge numbers of bugs in code that has 100% coverage in a test suite. The two are not the same!
- fit2rule 6y agoI'm not offering 100% coverage as anything but a level playing field, that things are tested. Also, code-path analysis is pretty much more advanced than you might think, especially when you get auto/generated testing rigs configured for particular tasks - and especially if you have a compiler to test, as well as all the (reproducible) applications it may produce ..