3 ms·
> The benefit is reduced testing effort. I wouldn't agree with this assessment. PBT has frequently surprised me, finding bugs that I never would have thought t
by chriswarbo 2y ago
> The benefit is reduced testing effort.
I wouldn't agree with this assessment. PBT has frequently surprised me, finding bugs that I never would have thought to test. It's not merely a tool for generating the unit tests we would have written anyway; it's a way to write down how we think the parts of a system interact, and have the machine show us that we're wrong. Crucially, it's quite hard to prove ourselves wrong, since it's hard to think up scenarios that we hadn't considered, so it's nice to have the machine help with that part.
> The drawback is that savings comes from sacrificing knowledge of exactly what has been tested.
I encourage you to use the statistics-gathering features that many (most?) PBT systems provide (e.g. functions with names like "collect", "label", etc.). Also, whilst preconditions/filtering can be useful in a pinch, it's usually a good idea for a property to construct data of the appropriate sort itself (e.g. using arbitrary arguments to fill in an appropriate template); rather than relying on a completely arbitrary generator to hit scenarios you care about.
- thehappyfellow 2y agoI think the surprise part in under-appreciated. The quicktest tests I write are frequently of the form “let’s spend 15 minutes and see what happens” kind of tests - and I almost always find bugs that way. The return on investment is bonkers. I do think that immutable by default OCaml + good PBT tooling there helps a lot.