3 ms·
The problem with your hierarchy is that there's no empirical evidence supporting it. Small unit tests have not empirically been shown to have benefits over inte
by naasking 3y ago
The problem with your hierarchy is that there's no empirical evidence supporting it. Small unit tests have not empirically been shown to have benefits over integration tests, and test-driven design has failed to show a benefit over tests written after the fact. The only thing that seems to matter is that tests are written, and the more tests, the more chances of finding a bug. That's it. So your list is actually:
* integration and unit tests: since these are manually written, they scale poorly but are simple.
* property tests: since these are semi-automatic they scale better but are a bit more complicated to setup.
* fuzzing: almost fully automatic, although I don't differentiate this much from property-based testing.
* mutation based testing
- pfdietz 3y agoIs mutation based testing only better because it forces more tests to be written to kill the mutants? Also, mutation based testing is really orthogonal to the others, since it's a way of evaluating the adequacy of the tests, not of actually testing. One could easily imagine using PBT/fuzzing to generate (and then simplify) tests with the express goal of killing mutants.
- eru 3y ago> Small unit tests have not empirically been shown to have benefits over integration tests, [...] Do you have any links to these studies? > * property tests: since these are semi-automatic they scale better but are a bit more complicated to setup. Once you are in the groove, I find property based tests to be simpler (or not harder) than example based tests. But that's when I am writing tests as I am developing the system, ie I take testability into account in the design.