4 ms·
This is a perfect example of why line/function coverage is a silly measurement. It doesn't take into account global state and function input. Take the second fu
by Too 12y ago
This is a perfect example of why line/function coverage is a silly measurement. It doesn't take into account global state and function input. Take the second function above, you could get 100% line coverage in just one run. But the function does exactly the same thing as the one where you only got 50% coverage, already things smell fishy. You can also test the function with millions of different inputs, all of them giving the same 100% coverage, and it will work perfectly fine, until suddenly you try addOne(int.max) and it will fail.
What you want to be really sure is state coverage, or input range coverage assuming your functions are pure. Now testing every function from int.min-int.max might seem unrealistic but what you have to do then is constrain the possible range of input or divide into ranges with special cases that you can somehow group together. Say for example int.min, negative numbers, zero, positive numbers and int.max.
Also, just because you covered a line doesn't mean it's correct, the only thing you've really tested is that the program doesn't crash. For the test to be really useful you also need a correct result of the output, added by a human. You can't just randomize input to increase the coverage.
- asveikau 12y agoI came to this thread hoping somebody would say pretty much exactly your first paragraph. The author appears to be selling a code coverage tool so of course he falls into the trap. In real life you have to remember that hitting a line once is not the same as showing it to be correct. People who buy into code coverage tools make this mistake a lot.