4 ms·
These aren't unit tests, they're generative tests. They're used for different things. Unit tests make assertions about how a program responds to a specific inpu
by lbarrow 11y ago
These aren't unit tests, they're generative tests. They're used for different things. Unit tests make assertions about how a program responds to a specific input. Generative tests make assertions about invariants in a program over a wide range of inputs.
The workflow for using them is very different. A unit test suite contains a finite number of assertions and, assuming no bugs, should run in a relatively small (or at least bounded) amount of time. A generative test, however, usually can run forever _by design_.
A typical workflow is to start a run overnight and see if it caught anything in the morning. If the generative suite finds any bugs, you turn the specific cases that caused the failures into unit tests and commit them.
- llimllib 11y agoSee also: quickcheck[1], hypothesis[2], ScalaCheck[3], etc [1]: http://www.cse.chalmers.se/~rjmh/QuickCheck/manual.html http://www.cse.chalmers.se/~rjmh/QuickCheck/manual.html [2]: https://hypothesis.readthedocs.org https://hypothesis.readthedocs.org [3]: http://www.scalacheck.org/ http://www.scalacheck.org/
- markc 11y agohttps://github.com/clojure/test.check https://github.com/clojure/test.check
- markbnj 11y agoSo is generative testing the same thing as fuzzing?
- platz 11y agoFuzzing: "The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leak" Fuzzing is more crude in that it doesn't really know what happens after providing the random value, it just tries to crash or fail asserts. generative testing involves writing an invariant, in the test itself, that holds over the random values. here are some examples of properties or invariants http://fsharpforfunandprofit.com/posts/property-based-testing-2/ http://fsharpforfunandprofit.com/posts/property-based-testin...