3 ms·
When a bad input is found, it gets saved to your corpus and gets checked on all subsequent runs. So re-running the test won't make it go away.
by icholy 5y ago
When a bad input is found, it gets saved to your corpus and gets checked on all subsequent runs. So re-running the test won't make it go away.
- Seb-C 5y agoWhich contravenes a fundamental good practice of testing: Each test should run independently and be stateless. Also, this would not work with most (all?) CI runs (which are inherently stateless), and working around this would be complex and introduce a lot of negative side effects.
- eru 5y agoWell, the alternative point of view is to see that when you have an infinite number of tests, you can't run them all, and need to sample them. (Saving the seeds to the corpus still leaves your tests independent and stateless. You are just sampling from your space of all possible tests a bit more intelligently in future.)