3 ms·

> They were just talking about test doubles / mocks above that. Those cause all sorts of risks with false positives and false negatives, for example by mocking

> They were just talking about test doubles / mocks above that. Those cause all sorts of risks with false positives and false negatives, for example by mocking out a function the code under test relies on, but the mock not being changed when the contract between those two pieces of code changes. The point is that it's absurd to try to characterize the regressions you're introducing as "false positive"/"false negative". Again, this is at best specious reasoning, and to me reflects that the blogger completely misses the point of having tests. Your code has behavior. Your tests verify invariants associated with this behavior. If you purposely change that behavior, it stands to reason that your task also involves updating how your code checks for that behavior. If you change behavior but fail to change how that behavior is checked, that is not a false positive or false negative. That represents a few problems you introduced: failing to update how that behavior is checked, and failing implement new behavior without checking if it works. Both reflect problems created by a developer failing to perform the basics of their work, and worse: blaming working tests for the mess he's creating. Think about the issue: if a highway administration decides to redesign an intersection but fails to move/update its stoplights, does this means the old stoplights cause accidents with their false positive greens?

Izkata 2y ago
> Your code has behavior. Your tests verify invariants associated with this behavior. If you purposely change that behavior, it stands to reason that your task also involves updating how your code checks for that behavior. If your tests don't fail before you update them, that's one false positive. Usually introduced by mocks that hide part of the behavior. If your tests were passing but a bug is discovered in the code that should have been caught by the given tests, that's another false positive. Also possibly introduced by mocks, but hopefully is more like just a missing assert. If your tests start failing with a correct code change and just straight need to be updated, that's a false negative (this is your example, halfway through the work). In either false positive case if it was caused by mocks, those are bad tests and should be rewritten and not just updated. > Think about the issue: if a highway administration decides to redesign an intersection but fails to move/update its stoplights, does this means the old stoplights cause accidents with their false positive greens? Lights at an intersection aren't tests, they're part of the working system. Tests in this example would be something more like a central reporting system that's supposed to show what's going on at the intersection. The system (the intersection and lights) can be working totally fine, but the reporting system is showing it messed up (false negative) or messed-up lights can be causing crashes but the reporting system shows it's working fine (false positive). In both cases it should be fixed, but the fix is independent of what - if anything - should be done with the intersection and lights themselves.

Izkata 2y ago

> Your code has behavior. Your tests verify invariants associated with this behavior. If you purposely change that behavior, it stands to reason that your task also involves updating how your code checks for that behavior. If your tests don't fail before you update them, that's one false positive. Usually introduced by mocks that hide part of the behavior. If your tests were passing but a bug is discovered in the code that should have been caught by the given tests, that's another false positive. Also possibly introduced by mocks, but hopefully is more like just a missing assert. If your tests start failing with a correct code change and just straight need to be updated, that's a false negative (this is your example, halfway through the work). In either false positive case if it was caused by mocks, those are bad tests and should be rewritten and not just updated. > Think about the issue: if a highway administration decides to redesign an intersection but fails to move/update its stoplights, does this means the old stoplights cause accidents with their false positive greens? Lights at an intersection aren't tests, they're part of the working system. Tests in this example would be something more like a central reporting system that's supposed to show what's going on at the intersection. The system (the intersection and lights) can be working totally fine, but the reporting system is showing it messed up (false negative) or messed-up lights can be causing crashes but the reporting system shows it's working fine (false positive). In both cases it should be fixed, but the fix is independent of what - if anything - should be done with the intersection and lights themselves.