4 ms·

Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actu

by ebg13 7y ago

Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing vs testing that the functions actually do what they're supposed to? If a function takes two ints and returns an int that is the integer division of the two, is the evaluation of exceptions and returned types not already implicit in the evaluation of the actual results? If you know that divide(a, b) should return c for sufficient candidates a, b, and c, then you _know_ that divide returns the right type without explicitly checking. And knowing that the divide function happens to return ints when given ints doesn't actually tell you that it's doing anything even close to the right behavior. So this both doesn't reduce the number of tests you need to write and is also obsoleted by actually writing the tests that you need.

timothycrosley 7y ago
By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of it, you can further specify what you expect from the result using the _auto_verify=Callable() parameter, or by looking at the individual test cases. Simple example: from my_library import add from hypothesis_auto import auto_pytest @auto_pytest() def test_add(test_case): add_result = test_case() if test_case.params.kwargs['number_1'] > 0 and test_case.params.kwargs['number_1']: assert add_result > test_case.params.kwargs['number_1'] assert add_result > test_case.params.kwargs['number_1']
- ebg13 7y ago
  > By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). So it's a fuzzer?
  - timothycrosley 7y ago
    Yes! A smart one though, that limits itself to the type constraints you've specified to avoid redoing the work of your type checker.
progval 7y ago
It's also useful to check the function doesn't crash. I use hypothesis to test a DB abstraction, and it often caught bugs in edge cases I didn't consider: timestamps/timezones too large, strings containing null bytes, etc. These would be caught by the tests generated by hypothesis-auto.

by ebg13 7y ago

timothycrosley 7y ago

By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of it, you can further specify what you expect from the result using the _auto_verify=Callable() parameter, or by looking at the individual test cases. Simple example: from my_library import add from hypothesis_auto import auto_pytest @auto_pytest() def test_add(test_case): add_result = test_case() if test_case.params.kwargs['number_1'] > 0 and test_case.params.kwargs['number_1']: assert add_result > test_case.params.kwargs['number_1'] assert add_result > test_case.params.kwargs['number_1']

ebg13 7y ago
> By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). So it's a fuzzer?
- timothycrosley 7y ago
  Yes! A smart one though, that limits itself to the type constraints you've specified to avoid redoing the work of your type checker.

progval 7y ago

It's also useful to check the function doesn't crash. I use hypothesis to test a DB abstraction, and it often caught bugs in edge cases I didn't consider: timestamps/timezones too large, strings containing null bytes, etc. These would be caught by the tests generated by hypothesis-auto.