3 ms·

Not sure if this is already mentioned but for me the most concise illustration of this fallacy was in The Pragmatic Programmer book. They had a function like

Not sure if this is already mentioned but for me the most concise illustration of this fallacy was in The Pragmatic Programmer book. They had a function like this: double f( double x ) { return 1/ x; } They pointed out that it is trivial to get 100% coverage in test cases but unless your tests include passing in 0 as the parameter you are going to miss an error case.

dullgiulio 9y ago
Exactly, or to put it another way: you are not forced to cover the code you didn't write; in this case, the non-zero check before the division. Mix this with unchecked exceptions, mocking of any real-world interaction that will generate errors in productions, and testing becomes a cargo-cult of quality.
voidpointer 9y ago
passing 0 to f returns infinity (IEEE 754) Not arguing the point here. It's just a terrible example.
- LgWoodenBadger 9y ago
  I thought you'd get NaN, not Inf. Of course I'm probably misremembering.
  - johnwatson11218 9y ago
    I think what the above returns depends on the language. In java I think there is an actual run time exception for this. The point of the original comment is to show how 100% coverage is easy to achieve but often meaningless. There is some other more abstract concept related to the classes of input that can possibly be passed to a method. IMHO 100% coverage and "test first" have done more harm than good to the cause of automated testing.
- bluGill 9y ago
  You are both correct and wrong: I want X/0 to immediately crash (fatal error) before the bad data that caused my to try to divide by zero gets propagated farther. Now that I think about it, I really want X/Y where Y is "close" to 0 to crash too. Of course I made the above up on the spot, but it is a reasonable thing to do for some situations. Those who know floating point math are well aware that dividing by something close to zero tends to result in very large errors, vs the true answer. (particularly if the division is part of a larger calculation)

dullgiulio 9y ago

Exactly, or to put it another way: you are not forced to cover the code you didn't write; in this case, the non-zero check before the division. Mix this with unchecked exceptions, mocking of any real-world interaction that will generate errors in productions, and testing becomes a cargo-cult of quality.

voidpointer 9y ago

passing 0 to f returns infinity (IEEE 754) Not arguing the point here. It's just a terrible example.

LgWoodenBadger 9y ago
I thought you'd get NaN, not Inf. Of course I'm probably misremembering.
- johnwatson11218 9y ago
  I think what the above returns depends on the language. In java I think there is an actual run time exception for this. The point of the original comment is to show how 100% coverage is easy to achieve but often meaningless. There is some other more abstract concept related to the classes of input that can possibly be passed to a method. IMHO 100% coverage and "test first" have done more harm than good to the cause of automated testing.
bluGill 9y ago
You are both correct and wrong: I want X/0 to immediately crash (fatal error) before the bad data that caused my to try to divide by zero gets propagated farther. Now that I think about it, I really want X/Y where Y is "close" to 0 to crash too. Of course I made the above up on the spot, but it is a reasonable thing to do for some situations. Those who know floating point math are well aware that dividing by something close to zero tends to result in very large errors, vs the true answer. (particularly if the division is part of a larger calculation)