5 ms·
Let's write unit tests for unit tests!
by sgt101 6y ago
Let's write unit tests for unit tests!
- solarengineer 6y agoWhen that starts to happen, it’s time to review what exactly one is Test Driving.
- aurelianito 6y agoI view the tested code as the test of the tests.
- ThePadawan 6y agoTo be fair, I worked at a company that had CI for running unit tests so nothing could be merged into remote master until the build was green. The build asserted that the number of failing unit tests was 0. It turns out that "breaking the unit test library completely such that 0 tests run instead of all thousands of them" did not break the build. When someone noticed a week later, that was not fun to clean up.
- rwbhn 6y agoRecently added an assertion that the # of tests succeeding was > 0 to cover a similar issue.
- the_af 6y agoTests that test nothing useful are surprisingly easy to write, and lead to a similar result -- even in cases where you do run them! That's why it makes some sense to test the tests (or at least, check their bug-detecting quality).
- Tainnor 6y agoThat's why you should IMHO always make sure you see your test fail at least once. TDD gives you that for free, but you don't have to use TDD: just manually breaking your implementation to see if the test breaks in the way it's supposed to should work equally well (bonus points if you make sure the error message is also precise enough to debug the issue).
- the_af 6y agoYes. But just like we automate tests instead of manually testing -- that's what computers are for after all: automation -- we can automate the tests of tests! Also, tests that once made sense might have become useless, so it seems relevant to have a way of automatically detecting obsolete "this doesn't test anything" tests even though they could have been useful once. Without relying too much on human intervention, that is.
- Tainnor 6y agothat's basically mutation testing but I've never seen a place that has been able to use it... maybe at some point the tools will get better
- je42 6y agoso, on that topic: - most test frameworks use unittests to test themselves. - tests need to have value. the dev/team/organization needs to come up with the sensible measure of value that guides poeople what to test and how to test it.
- the_af 6y ago> Let's write unit tests for unit tests! Tests for tests exist. One such technique is called "mutation testing" [1]. I don't know of any job that actually uses them, but they make sense to me: the tests need checks too, otherwise how do you know they are adding value? It's too easy to write worthless tests, see them pass, and get a false sense of security. Techniques such as mutation testing check how sensitive your tests are to actual bugs (or program changes): do they detect them? If not, they aren't useful. [1] https://en.wikipedia.org/wiki/Mutation_testing https://en.wikipedia.org/wiki/Mutation_testing