4 ms·

How do you do shrinking? IMO that's the property testing killer feature

• gamegoblin 2y ago
  That's why my outer loop goes from low-complexity cases to high-complexity cases, it has basically the same effect as shrinking (without actually having to do any work)
  • cpeterso 2y ago
    Like a breath-first search instead of depth-first so you hit the shorter path first?
    • gamegoblin 2y ago
      Yes exactly, though not exhaustive, so more like beam search actually
• josephg 2y ago
  I do this sort of thing too. I don’t have an automated way to shrink my input - but that’s usually fine in practice. For example, say I’m testing a data structure. I’ll have an outer loop that picks a seed and an inner loop that does ~100 mutations of a data structure instance, testing assertions each time. If there’s a failure, I’ll try a bunch of seeds to look for one that fails the fastest. (In the fewest possible inner loop iterations). It’s not a perfect system, but for most bugs I can usually get a test case that only needs 5-10 steps before a crash occurs. And if the reduction step finds a different bug? That’s no problem at all. I fix what I found and go back to trying more seeds.