3 ms·
It's important to remember that our own mental model is a very important part of any project. I've often read complaints about people tracking down a test failu
by chriswarbo 6y ago
It's important to remember that our own mental model is a very important part of any project. I've often read complaints about people tracking down a test failure and finding out it's actually a problem with the test and not the implementation, as if that were a waste of time. However, we need to ask the question: why did someone write that test, and do it in that way? If the answer is "because they didn't understand how the system works" then we've just exposed a bug in someone's mental model (often our own). It's not a waste of time to find and fix these failures (either by changing the test, now that we've improved our understanding of the system; or by removing it as superfluous). The article's example of checking for the round-robin algorithm rather than a lack of race conditions seems like one of these cases.
Of course, this isn't the only reason why supposedly-spurious test failures occur. If the answer to "why did someone write this test in this way?" is something like "to increase code coverage" or "they watched a talk that said to do things this way" or something equally silly then that's harder to spin as a positive. Hopefully it might at least expose failures in those development practices?
Unfortunately the developer who has to fix these failures is often someone who already understands, and wouldn't have written the test that way. Perhaps it's exposing a problem in their documentation, coding guidelines, etc.?