4 ms·
The point is that there is a limit to testing. And some people go way overboard with it. You'll never get 100% coverage. It's simply not possible. Now, that do
by w8rbt 10y ago
The point is that there is a limit to testing. And some people go way overboard with it. You'll never get 100% coverage. It's simply not possible.
Now, that doesn't mean you should not test. It means you should understand the limits of unit testing and test what is important as best as you can. Most every software engineering class at universities will cover this in-depth.
- adrianN 10y agoOf course you get 100% coverage if that's your goal. SQLite for example is a big project with 100% coverage: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html If you want rock solid software you need to spend the time to properly test it.
- cleeus 10y agowell, not at all times. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3416 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3416
- adrianN 10y agoGreat, you've proved that 100% test coverage doesn't mean 0% bugs. The actual question is whether there is a significant difference in reliability in software with 80% test coverage compared to software with 100% coverage.
- cleeus 10y agoat which point we first need to talk about the type of coverage ... tree coverage or line coverage (or something else?)
- icebraining 10y agoIn the case of SQLite, it's 100% branch coverage and 100% MC/DC coverage: https://www.sqlite.org/th3.html https://www.sqlite.org/th3.html
- mikmoila 10y agoYup and also, how about missing code? Even 100% isn't enough :)
- strictfp 10y agoAgreed. 100% line coverage doesn't mean that all functionality is tested; You don't test the full range of values every variable can take on, all possible throw-catch pairs etc. So the question is if the difference between 80% and 100% line coverage is that big. In fact, I think it can be decieveing to use the term '100%'. Arguably the reability of your program can be increased from 100% line coverage by adding an extra non-tested value check, effectively lowering the coverage.
- huherto 10y agoIndeed there are diminishing returns as you increase coverage. I personally shoot to go over 80%. You may try to do 90% but it is going to significantly increase the development cost. You end up writing a whole unit test just to test one trivial line.
- Nomentatus 10y agoMore precisely, there's a very strong power-law re testing; therefore the frequency of testing is also very important. You don't want to test just to create QC-Theatre, that's just narcissism. But most of us err on the side of laziness. The empirical data doesn't say Agile works, it says frequent testing works.