4 ms·
Yes, timeouts are bad, but that’s true even for regular tests with determininistic input. You’ll get flakes running on different machines, depending on how much
by skybrian 1y ago
Yes, timeouts are bad, but that’s true even for regular tests with determininistic input. You’ll get flakes running on different machines, depending on how much load is on them.
To test an entire system with reproducable failures, you probably need something more heavyweight like Antithesis. Property tests are more useful for unit tests.
- crabbone 1y agoMultiple-step unit tests are easier to modify to only slightly change the behavior, which is usually too hard to do with property-based testing where small changes to initial input cause huge changes downstream. But, in general, yes, unit test failure with many steps would've been just as difficult to interpret. My experience though was that once such a difficult failure is encountered during property-based testing, one has to write a unit test to reproduce the error anyways. But it's hard to assess the probability of the unexpected behavior of being an actual bug. Sometimes you discover that the system behavior was underspecified, or that you misunderstood how the system is supposed to behave after reading the specification.