4 ms·
> Can you explain a bit more? Not the parent commenter, but i'll attempt to chime in: in addition to the default "dopey" practice of measuring coverage in term
by shoo 5y ago
> Can you explain a bit more?
Not the parent commenter, but i'll attempt to chime in: in addition to the default "dopey" practice of measuring coverage in terms of statements of code, in addition to data coverage as mentioned by the parent, you could also consider coverage in terms of possible execution paths through the code or coverage in terms of requirements. For interesting computational code (i dunno, consider a mixed integer program solver) it might be possible to get 100% statement or 100% branch coverage at the same time as getting approximately 0% data coverage, 0% execution path coverage, and 0% coverage of functional requirements.
Another perspective is how much value the test gives relative to the effort and cost to set it up and maintain it. You can have 100% unit test coverage (statement, branch, data, whatever) but maybe your application doesn't even do anything when it boots, because you forgot to call any of the units. So if effort and time is limited, you might get more return on investment from writing some good integration tests or hooking up a fuzzer than focusing on hitting the somewhat arbitrary but easy to measure 100% statement coverage number.
Also, like everything in life, focusing on a single objective like "statement coverage" and trying to maximise it can lead to strange outcomes. There was a fascinating blog post / war story somewhere describing a software system architecture where the organisation mandated non-negotiable minimum 80% statement coverage for unit test suites. This system had some pretty low level code and a bunch of that code wasn't possible to test in unit tests. So how do you hit the target? Well, if you add a few useless layers to your application architecture that are easy to unit test, then you test them and leave the hard code that interfaces with the real world / hardware untested, and you hit your mandated target, but the overall level of software quality is worse.
- satisfice 5y agowell said