3 ms·

I've been asked many times what is the right code coverage percentage to aim for. I've also asked this many times in interviews, to tease out a discussion. My a

I've been asked many times what is the right code coverage percentage to aim for. I've also asked this many times in interviews, to tease out a discussion. My answer is: you need full coverage. I then continue to explain that I didn't say 100%. Although I believe that the average Java microservice (which is the general sphere that I move around) can easily achieve >98% coverage. Easily. But "full coverage" means that all reasonable scenarios should be covered. No code coverage tool is going to measure that for you (I think). Reasonable scenarios are... everything? Time is finite, so most teams will never be exhaustive, but one can try. The "80% coverage crowd" is throwing in the towel before even starting.

saalweachter 5y ago
I've occasionally been lulled into a false sense of security by the difference between "every line is run" and "the function is fully tested". On the other hand, even if I'm half-assing the unit tests, and implementing the bare minimum necessary to reach whatever arbitrary coverage% is being sought ... I still find bugs, bugs that almost certainly would have a real impact in production, even if I already ran an end-to-end test and was pretty sure it was working fine[*]. Not every time I write tests, but often enough to feel like it pays for all of the time "wasted". So even if coverage metrics maybe arbitrary, I don't really begrudge having to meet them. And honestly, once you're forced to start writing the tests just for coverage, it isn't that much extra work to make them a little more comprehensive than the floor set by coverage. [*] Honestly I'm shocked at how well buggy software works, for the most part.
- AlexCoventry 5y ago
  That's my experience, too. Coverage isn't a good proxy for test quality, for sure, but it's a good proxy for having looked carefully at every line. I think coverage testing is useful as long as you're prepared to throw those tests away when you refactor, or at least ignore them during refactoring.
jamesfinlayson 5y ago
Agreed - if you can test it then test it. 100% coverage may not be easily achievable with some languages/frameworks, but as other have pointed out, it's a good idea to work towards it for dynamic languages so you can reduce the possibility of runtime errors.

saalweachter 5y ago

I've occasionally been lulled into a false sense of security by the difference between "every line is run" and "the function is fully tested". On the other hand, even if I'm half-assing the unit tests, and implementing the bare minimum necessary to reach whatever arbitrary coverage% is being sought ... I still find bugs, bugs that almost certainly would have a real impact in production, even if I already ran an end-to-end test and was pretty sure it was working fine[*]. Not every time I write tests, but often enough to feel like it pays for all of the time "wasted". So even if coverage metrics maybe arbitrary, I don't really begrudge having to meet them. And honestly, once you're forced to start writing the tests just for coverage, it isn't that much extra work to make them a little more comprehensive than the floor set by coverage. [*] Honestly I'm shocked at how well buggy software works, for the most part.

AlexCoventry 5y ago
That's my experience, too. Coverage isn't a good proxy for test quality, for sure, but it's a good proxy for having looked carefully at every line. I think coverage testing is useful as long as you're prepared to throw those tests away when you refactor, or at least ignore them during refactoring.

jamesfinlayson 5y ago

Agreed - if you can test it then test it. 100% coverage may not be easily achievable with some languages/frameworks, but as other have pointed out, it's a good idea to work towards it for dynamic languages so you can reduce the possibility of runtime errors.