3 ms·

Table-based testing is property-based testing. Nothing in property testing says who has to choose the values used. There are libraries that automatically test

Table-based testing is property-based testing. Nothing in property testing says who has to choose the values used. There are libraries that automatically test valid/invalid inputs for types. But automatic case generation is not required for property based testing. A refinement is pairwise testing, where the machine chooses some minimal subset of cases to span the test space to combat combinatoric explosion and run in a reasonable amount of time. https://www.pairwise.org https://www.pairwise.org Further refinement is possible to cut the run time by pruning the cases-to-be-run based on previous failures. Unfortunately, then things start to seem even more "magical" and there is no set number of test cases always run and non-technical (or non-testing) management tends to have problems with that. One day we'll have nice things . . .

IanCal 3y ago
I've never heard of parameterised tests being called property based testing - they don't define a property just a list of cases. At that point we may as well call all unit tests property based tests and lose any useful distinction.
- eru 3y ago
  I agree that calling table based testing 'property based testing' is stretching the definition. > Nothing in property testing says who has to choose the values used. I take issue with that. Properties of interest are some variant of 'for all x from some set s, some property holds.' Then instead of testing all possible x, we randomly sample. But it is important that our tests define the set s. When we do example based testing (whether traditional unit testing, or table based testing), the set s is usually only implied. So you might be testing your functionality for strings 'foo', and 'bar' and perhaps 'foo bar'. But it's not clear, whether you also mean it to pass for the string '' or '안녕하세요' or '\t\n'. That's at best implied, or perhaps vaguely described in a human readable comment.
eru 3y ago
There's some efforts to guide test generation for property based testing to make the instruction pointer explore as large a space as possible. This effort is more mature in the fuzzing community. See eg American Fuzzy Lop https://github.com/google/AFL https://github.com/google/AFL

IanCal 3y ago

I've never heard of parameterised tests being called property based testing - they don't define a property just a list of cases. At that point we may as well call all unit tests property based tests and lose any useful distinction.

eru 3y ago
I agree that calling table based testing 'property based testing' is stretching the definition. > Nothing in property testing says who has to choose the values used. I take issue with that. Properties of interest are some variant of 'for all x from some set s, some property holds.' Then instead of testing all possible x, we randomly sample. But it is important that our tests define the set s. When we do example based testing (whether traditional unit testing, or table based testing), the set s is usually only implied. So you might be testing your functionality for strings 'foo', and 'bar' and perhaps 'foo bar'. But it's not clear, whether you also mean it to pass for the string '' or '안녕하세요' or '\t\n'. That's at best implied, or perhaps vaguely described in a human readable comment.

eru 3y ago

There's some efforts to guide test generation for property based testing to make the instruction pointer explore as large a space as possible. This effort is more mature in the fuzzing community. See eg American Fuzzy Lop https://github.com/google/AFL https://github.com/google/AFL