5 ms·
How do you know that your tests are useful ? most (I'm a senior tester so I know a few) teams write tests that are useful when they are new or when a new featur
by 2rsf 6y ago
How do you know that your tests are useful ? most (I'm a senior tester so I know a few) teams write tests that are useful when they are new or when a new feature is being developed, those tests quickly stop finding problems and in the worst case give false negatives (fail for no reason).
A related problem are Rotten green tests [1]
[1] https://dl.acm.org/doi/10.1109/ICSE.2019.00062 https://dl.acm.org/doi/10.1109/ICSE.2019.00062
- codeulike 6y agoIf a test never fails unexpectedly then you know that particular test has been a waste of time. e.g. if no regressions ever get picked up by the test subsequently
- rimliu 6y agoThat's an interesting statement. Imagine I have a test which never fails unexpectedly. But it expectedly fails when I decide to refactor some code. By your definition that test was a waste of time, by my definition it did excatly what it was supossed to do.
- codeulike 6y agoIf you're expecting it to fail then your brain is already doing the work that the test is meant to do. It serves as an expensive to-do reminder in that scenario. If you are part way or all the way through your refactor and think you're on the right track and then you unexpectedly have a failing test, then I would say that that was a useful bit of test code.
- exdsq 6y agoThe point they're trying to make, I believe, is that you should write tests that'd pass mutation testing (testing the tests, so to speak). https://en.wikipedia.org/wiki/Mutation_testing https://en.wikipedia.org/wiki/Mutation_testing