3 ms·

I think the important thing here is the change: 100 -> 95. A 5% decrease should be noticed, in some fashion. You don't need to be at 100% coverage to do that.

• yummyfajitas 15y ago
  The change is important, but so is the absolute number of untested lines. A) Untested lines of code: 10-25 B) Untested lines of code: 2-4, 10-25, 54-68, 59, 63, 68, 71-77, 85 I know that in case A, I'm far more likely to look at 10-25 and come up with a good test case. In case B, I'm not sure I want to skim 2-4 (skip, it's ok), 68 (just a __unicode__), 59 (oh, that should never be executed), ah, 10-25 is what I should look at. It's sort of the broken windows theory applied to testing. That said, I don't have 100% test coverage in most of my projects.
  • pbiggar 15y ago
    A tool which told you there was an absolute change of 5%, but didnt tell you where that change was, would not be all that useful. I'm sympathetic to the "all-or-none" broken windows philosophy that many developers use, but I disagree with it. If you need to get to 100% of some metric in order to get some visibility on its changes, you're just not going to get that metric. At Circle, we make hosted CI, and I frequently talk to customers who use jenkins or something, and whose tests are broken. Inevitably, they broke the tests one day, their priorities dictated that they couldn't fix it right away, so they left their test suite red. Because it was red, they didn't notice the suite getting more and more broken over time. If their test suite hadn't been binary, they would have learned immediately when they broke more tests, and they would have been in a much better place. TL;DR I blame the tools.