2 ms·

A lot of people here seem to have strong opinions against 100% coverage, so I'll risk their ire with my strong opinion in favor. If you have, say, 95% coverage

by biztos 10y ago

A lot of people here seem to have strong opinions against 100% coverage, so I'll risk their ire with my strong opinion in favor. If you have, say, 95% coverage -- and most corporate dev orgs would be thrilled with that number -- and then you commit some new code (with tests) and are still at 95%, you don't know anything about your new code's coverage until you dig into the coverage report. Because your changes could have had 100% coverage of your new thing but masked a path that was previously tested; or had 10% but exercised some of the previously missing 5%. If you have 100% coverage and you stay at 100% then you know the coverage of your new code: it's 100%. Among other things this lets you use a fall in coverage as a trigger: to block a merge, to go read a coverage report, whatever you think it warrants. Also, as has been noted elsewhere, anything other than a 100% goal means somebody decides what's "worth" testing... and then you have either unpredictable behavior (what's obvious to whom?) or a set of policies about it, which can quickly become more onerous than a goal of 100%. It's important to remember that the 100% goal isn't going to save you from bad tests or bad code. It's possible to cheat on the testing as well, and tests need code review too. There's no magic bullet, you still need people who care about their work. I realize this might not work everywhere, but what I shoot for is 100% coverage using only the public API, with heavy use of mock classes and objects for anything not directly under test and/or not stable in real life. If we can't exercise the code through the public API's then it usually turns out we either didn't rig up the tests right, or the code itself is poorly designed. Fixing either or both is always a good thing. I don't always hit the 100% goal, especially with legacy code. But it remains the goal, and I haven't seen any convincing arguments against it yet. Open the flame bay doors, Hal... :-)

Hermel 10y ago
The problem with the "100%" goal is that it provides a false sense of safety. There can still be plenty of bugs in code that has "100%" test coverage. The 100% are an illusion. To truly get to 100%, it does not suffice that every line of code is executed, you must also test all valid inputs - which is utopic.
- biztos 10y ago
  I concede that for people who don't understand what unit testing does, 100% might provide a false sense of security. But I don't think that's a problem with the goal... that would be like saying the problem with staying fit is that it provides a false sense of longevity, whereas you could still get brain cancer or be hit by a car. If you know what's going on -- and hopefully you do if you're making decisions like whether to pursue a 100% coverage goal -- then your point is (I hope) self-evident. It's also why code review is necessary and not just the coverage metric. 100% coverage is not "100% of all things that could happen," it's "100% of all code execution paths" and (hopefully) a sample of all known types of valid and invalid inputs. Anyway, so far I've found that most developers respond pretty well to the 100% idea if you explain its utility and what it does and doesn't get you. From a ridiculously small sample, granted. The utopian risk, I think, is much more around the hope that people high up the food chain in your organization actually understand this stuff. Consider how many VPs of Engineering did a year or two of engineering work (before unit testing was in vogue) then got that MBA and have been busy taking meetings ever since.

by biztos 10y ago

Hermel 10y ago

The problem with the "100%" goal is that it provides a false sense of safety. There can still be plenty of bugs in code that has "100%" test coverage. The 100% are an illusion. To truly get to 100%, it does not suffice that every line of code is executed, you must also test all valid inputs - which is utopic.

biztos 10y ago
I concede that for people who don't understand what unit testing does, 100% might provide a false sense of security. But I don't think that's a problem with the goal... that would be like saying the problem with staying fit is that it provides a false sense of longevity, whereas you could still get brain cancer or be hit by a car. If you know what's going on -- and hopefully you do if you're making decisions like whether to pursue a 100% coverage goal -- then your point is (I hope) self-evident. It's also why code review is necessary and not just the coverage metric. 100% coverage is not "100% of all things that could happen," it's "100% of all code execution paths" and (hopefully) a sample of all known types of valid and invalid inputs. Anyway, so far I've found that most developers respond pretty well to the 100% idea if you explain its utility and what it does and doesn't get you. From a ridiculously small sample, granted. The utopian risk, I think, is much more around the hope that people high up the food chain in your organization actually understand this stuff. Consider how many VPs of Engineering did a year or two of engineering work (before unit testing was in vogue) then got that MBA and have been busy taking meetings ever since.