3 ms·
> Another issue is that randomisation introduces flakiness when the input space is large. But that is precisely the point of property-based testing - as oppose
by mrkeen 3y ago
> Another issue is that randomisation introduces flakiness when the input space is large.
But that is precisely the point of property-based testing - as opposed to hard-coded inputs. The flakiness is in your code, not your test. Change your code until the PBT stops flaking.
- GrumpySloth 3y agoDoor A: tests find bugs sometimes (property-based) Door B: tests find bugs almost always with a fraction of the code (my approach) I choose Door B. I still test properties tbh. I just don’t let randomness ruin it and don’t need to implement any shrinking.
- bluGill 3y agoThis is not an exclusive choice. Choose all doors as each will gike you something the others don't. All includes things not yet invented.
- GrumpySloth 3y agoI have a finite budget of code and time I can afford to spend. One of the choices gives me a much better RoI in my experience. I prefer to spend the rest of the budget on the main code.
- eru 3y agoPython's Hypothesis automatically saves the random seeds that find counterexamples, so the next time you run your test suite, you can reproduce that bug.
- mrkeen 3y agoIf your code does the same thing every time it's run.