4 ms·

Test coverage is a fake feel-good metric that makes you want to increase the number and by this give you the impression of progress. Real progress is measured d

Test coverage is a fake feel-good metric that makes you want to increase the number and by this give you the impression of progress. Real progress is measured differently. Test as much as necessary, but not everything.

bphogan 16y ago
I disagree. Test coverage is a wonderful metric. 100% coverage is a feel-good metric. I can easily achieve 100% coverage with bad tests. But I routinely use coverage as my guide to find things that might benefit from further testing, as in "oooh this path is never tested and stuff is more likely to break here". I try really really hard not to write useless tests-I use tests to drive the development of my app. I need feature x, so I write a test for it. I end up with relatively high coverage because of that. And because I've gotten more disciplined over the years, I'm only implementing the stuff that passes my tests, so I'm less likely now to have things that aren't covered. So, I view coverage as a guide, not a rule, but it's not a useless metric at all unless you designate an arbitrary percentage.
StavrosK 16y ago
I also disagree. I don't know about everyone, but on my projects (ones I've built by myself, so nothing huge) I can always give a probability of a bug existing in a part of the code. I've frequently written a feature and thought "yeah, this is going to break" only to find that it does in testing. When I write tests, I don't care (too much) about coverage as about writing good tests. If I'm confident that I've gotten most cases, new features are much easier to add, even if the coverage isn't that great. Covering a part of the code you know is unlikely to fail isn't a very good use of anyone's time, I think.
michaelfeathers 16y ago
We shouldn't underestimate the value of feel good metrics, though. Feeling good is important :-) I like to measure code coverage for changed code within a rolling time window, say 6 months. If people are writing tests for all they change, you rapidly get close to 100%. This gets past the issue that in many large code bases, you could probably write tests for ten years straight, end up writing lots of tests for code that doesn't change, and still not get close to 100%.

bphogan 16y ago

I disagree. Test coverage is a wonderful metric. 100% coverage is a feel-good metric. I can easily achieve 100% coverage with bad tests. But I routinely use coverage as my guide to find things that might benefit from further testing, as in "oooh this path is never tested and stuff is more likely to break here". I try really really hard not to write useless tests-I use tests to drive the development of my app. I need feature x, so I write a test for it. I end up with relatively high coverage because of that. And because I've gotten more disciplined over the years, I'm only implementing the stuff that passes my tests, so I'm less likely now to have things that aren't covered. So, I view coverage as a guide, not a rule, but it's not a useless metric at all unless you designate an arbitrary percentage.

StavrosK 16y ago

I also disagree. I don't know about everyone, but on my projects (ones I've built by myself, so nothing huge) I can always give a probability of a bug existing in a part of the code. I've frequently written a feature and thought "yeah, this is going to break" only to find that it does in testing. When I write tests, I don't care (too much) about coverage as about writing good tests. If I'm confident that I've gotten most cases, new features are much easier to add, even if the coverage isn't that great. Covering a part of the code you know is unlikely to fail isn't a very good use of anyone's time, I think.

michaelfeathers 16y ago

We shouldn't underestimate the value of feel good metrics, though. Feeling good is important :-) I like to measure code coverage for changed code within a rolling time window, say 6 months. If people are writing tests for all they change, you rapidly get close to 100%. This gets past the issue that in many large code bases, you could probably write tests for ten years straight, end up writing lots of tests for code that doesn't change, and still not get close to 100%.