4 ms·
There are a couple of useful things missing here at a glance. One is shrinking, where the library will attempt to find simpler cases of failures rather than jus
by thom 4y ago
There are a couple of useful things missing here at a glance. One is shrinking, where the library will attempt to find simpler cases of failures rather than just brute force a random one. Second is how to create related pieces of data that are constrained in some way, above and beyond pure random bytes, but I assume there’s some way here to manually interpret the random bytes. But sometimes you have to hand-tune these things because you don’t just want to wait for the RNG to guess the right combination to create valid data.
I’ve had some success with property based testing, at least for heavily algorithmic code. For example I wrote a kNN implementation after finding all sorts of edge cases in others (floating point shenanigans included). You can succinctly capture properties like: returning the right number of clusters, each point being closest to its cluster’s mean, each cluster being centred on its centroid, all points being assigned to a cluster, and all clusters being non-empty. But even then you could game things if you wanted to.
Where I’ve struggled is in the general case. Most tests don’t have good property based alternatives, beyond just repeating the logic of the code under test, which seems pointless. And so you have this constant tension of wondering if a particular test could or should be property based.