3 ms·
Do you consider coverage an effective metric? I've got some code that has a test suite which is effectively a bunch of low level driver checks, plus a bunch of
by joshvm 7y ago
Do you consider coverage an effective metric? I've got some code that has a test suite which is effectively a bunch of low level driver checks, plus a bunch of common example snippets and checks for eg empty inputs etc.
Coverage gives an idea of how many lines of code have been run, but obviously no guarantees of correctness for those specific lines (eg you can't detect a double negative).
It's worked well for me so far, since the important parts are (a) the hardware communication works and (b) users can process and output data in a way that is correct. No need to obsessively check the intermediate steps if the output is good.
- codeulike 7y agoI would agree that "coverage gives an idea of how many lines of code have been run, but obviously no guarantees of correctness for those specific lines"
- joshvm 7y agoPerhaps I should rephrase. If you're trying to effectively unit test a project, when do you decide you've tested enough? And are there any metrics which help support that? Coverage for example is a weak signal that you've at least run some fraction of your codebase at test time.
- codeulike 7y agoI just write tests for the bits that I know I'm going to have lots of trouble with. You can tell, after a while
- P_I_Staker 7y agoI definitely think coverage is a worthy metric to track. It can provide meaningful information about the "doneness" of your tests. It shouldn't drive testing though, and especially, you shouldn't write your tests specifically "to get coverage". Yes, lots of people do this in environments where "getting 100% coverage" is mandatory. That said, I've found issues specifically after targetting blocks for testing, which were highlighted by incomplete coverage. It's crucial to always remember that coverage is predicated on have good tests. At the very least every test must test something. Sounds obvious, however it's possible to get 100% coverage with a single test, test nothing, and still miss issues.
- Izkata 7y ago> which were highlighted by incomplete coverage This is the single most important part of code coverage IMO: we don't care about what the tests cover, we care about what the humans never considered. For this reason I'm a proponent of 100% coverage with a major caveat: Any code you explicitly decided not to test gets marked "no cover", so it doesn't count for or against the coverage score. This way branches that were accidentally missed really stand out, and we're not bogged down by having to test 100% of the code.
- P_I_Staker 7y agoGreat idea we'll just add // Unit Testing begin not covered // Unit Testing end not covered To the start and end of every file (or however it's done in your suite). Then we can leave early, go to the bar, and have a nice pint ;)