3 ms·

"these assertions hold for all inputs" Which would be great if it wasn't too computationally expensive. Most of the time, it just becomes generating a subset o

by _hilro 5y ago

"these assertions hold for all inputs" Which would be great if it wasn't too computationally expensive. Most of the time, it just becomes generating a subset of random (within range) input values to be tested thus decreasing the utility. We need better structured code such that each tests can be tied to specific code - code doesn't change - test doesn't re-run. That is way too hard and too risky using current tech and unit testing doesn't solve the problem.

chriswarbo 5y ago
> "these assertions hold for all inputs" > Which would be great if it wasn't too computationally expensive. Most of the time, it just becomes generating a subset of random (within range) input values to be tested thus decreasing the utility. Those are complaints about the testing/checking process; I was talking about test/property definitions. In particular, most of the properties we care about are universal ("for all"), but unit tests cannot even state those properties, let alone test them, let alone prove them. Property-based testing lets us state universal properties, and it lets us test them (although it cannot prove them). Those are two advantages compared to unit/functional tests. Regarding the actual checking process, and the cost/benefit it provides, that varies depending on the project's needs, the tools/frameworks used (e.g. random generation versus enumeration versus on-demand generation, etc.), the way testing is implemented (e.g. constructive generation versus filtering, domain-specific types versus general-purpose language builtins, etc.). Personally, property-based testing is my default approach (since "normal" unit/functional tests are simply parameterless property tests); I've done it in Haskell, Javascript, Python, Scala, Isabelle/HOL and PHP; and it's found all sorts of issues with my code that I would never have thought to write tests for.

by _hilro 5y ago

chriswarbo 5y ago

> "these assertions hold for all inputs" > Which would be great if it wasn't too computationally expensive. Most of the time, it just becomes generating a subset of random (within range) input values to be tested thus decreasing the utility. Those are complaints about the testing/checking process; I was talking about test/property definitions. In particular, most of the properties we care about are universal ("for all"), but unit tests cannot even state those properties, let alone test them, let alone prove them. Property-based testing lets us state universal properties, and it lets us test them (although it cannot prove them). Those are two advantages compared to unit/functional tests. Regarding the actual checking process, and the cost/benefit it provides, that varies depending on the project's needs, the tools/frameworks used (e.g. random generation versus enumeration versus on-demand generation, etc.), the way testing is implemented (e.g. constructive generation versus filtering, domain-specific types versus general-purpose language builtins, etc.). Personally, property-based testing is my default approach (since "normal" unit/functional tests are simply parameterless property tests); I've done it in Haskell, Javascript, Python, Scala, Isabelle/HOL and PHP; and it's found all sorts of issues with my code that I would never have thought to write tests for.