3 ms·
My main issue with that approach is that your tests become flaky due to random data generation in your tests. They can be fine in your staging build pipeline an
by bossyTeacher 2y ago
My main issue with that approach is that your tests become flaky due to random data generation in your tests. They can be fine in your staging build pipeline and fail at your prod build pipeline. Extensive logging doesn't change that fact.
- chriswarbo 2y agoYou can fix the PRNG seed in your build pipelines, which should make them execute the same examples for each test. Hypothesis has a nice feature which uses a hash of each test's source code as the PRNG seed for that test, which ensures the same examples are checked every time (unless someone changes the text of that particular test). It's a nice idea, but might be hard to implement in compiled languages (like Haskell, for example).