2 ms·

As others have said, code coverage is an extremely misleading statistic. Most tools measure code coverage by the number of lines of code that were executed in

As others have said, code coverage is an extremely misleading statistic. Most tools measure code coverage by the number of lines of code that were executed in a test. I can easily write "tests" with no expectations that hit every line of code. It is merely showing that in the cases the programmer thought of, it doesn't crash. Consider this as well: many interpreted languages count the definition of a method/function as a "covered line of code". In other words, if a method/class is defined then you have at least one (maybe several) lines of code "covered". Add to that things like declaring variables and you can easily get to a point where (in good code with very short methods) you have 60% coverage if the method is simply defined and run. If you are writing really good OO code that has very few conditionals, then you may get very, very close to 100% test coverage simply by running each method. Also, tests are code too. You are just as likely to write a bug in your test as you are in your production code. In fact, I've noticed that in some systems, I actually end up writing more lines of test code than production code. I might actually be more likely to write a bug in the tests. Especially for bugs that are due to problems with requirements, you are very, very likely to put the same bug in the tests as you are the production code. This is why I don't like calling these things "tests". IMHO the very best "tests" document assumptions that the original programmer had. If you break the assumption, the the test should break. It may or may not be a bug, but it is a big hint that you should be thinking very carefully about what you are doing. Code coverage is a good statistic in the reverse. It doesn't tell you much if you have good code coverage. On the other hand, it can lead you to find places where your tests are unlikely to be sufficient. Depending on the language/environment you use, if you have less than 80% code coverage in an area, it might indicate that nothing of value is being tested. It is a good place to start looking to check. Note that setting a code coverage target will defeat this strategy, so I recommend never, ever doing it. As I said, getting nearly 100% code coverage without actually testing anything is relatively trivial if the code is in any way decent (and if it isn't, then you have bigger fish to fry). Setting a threshold above the place where poor testing is obvious just means that you no longer have a means of finding places that are poorly tested.

aetherson 12y ago
I think that you think you're arguing with me, but are actually agreeing with me. Like, I agree with every claim about fact that you're making. Of course you can have high code coverage without having good tests. If all you do is, like, touch lines of code, all you're really testing is that they don't raise an exception. That's why I said that once you hit 80% code coverage or so, I start to want to know not "what your code coverage is," but "how good are your tests?" But if you don't even run the majority of your code, then your tests are kind of by definition... not testing your code well.

aetherson 12y ago

I think that you think you're arguing with me, but are actually agreeing with me. Like, I agree with every claim about fact that you're making. Of course you can have high code coverage without having good tests. If all you do is, like, touch lines of code, all you're really testing is that they don't raise an exception. That's why I said that once you hit 80% code coverage or so, I start to want to know not "what your code coverage is," but "how good are your tests?" But if you don't even run the majority of your code, then your tests are kind of by definition... not testing your code well.