4 ms·

Using Haskell's QuickCheck for Python

43 points by Russell91 12y ago

rusbus 12y ago
Or of course, you can skip haskell and just use the python port https://pypi.python.org/pypi/pytest-quickcheck https://pypi.python.org/pypi/pytest-quickcheck
- sa2ajj 12y ago
  It seems to provide random data generation only. I'd say _the_ feature of QuickCheck is minimisation of the counter example.
  - DRMacIver 12y ago
    Yep. This is why I wrote Hypothesis: http://hypothesis.readthedocs.org/ http://hypothesis.readthedocs.org/ / https://pypi.python.org/pypi/hypothesis https://pypi.python.org/pypi/hypothesis It's a lot more fully featured than the alternatives. It still lacks a few features of quickcheck (coarbitrary for function generation, labelling of examples), but it does example minimization and a whole bunch of things even quickcheck doesn't.
    - DRMacIver 12y ago
      Realised I should have linked to http://hypothesis.readthedocs.org/en/master/internals.html http://hypothesis.readthedocs.org/en/master/internals.html to back up that "a whole bunch of things even quickcheck doesn't" claim. :-)
    - deleted 12y ago
      [deleted]
    - teh 12y ago
      Hypothesis looks fantastic. A friend of mine and me spent a while on Google collecting Python QC implementations but didn't find any that implemented shrinking. Hypothesis escaped our search. I don't know how but the project could do with better Google ranking :) It has some really useful features like storing the counterexamples. The CLA is a bit odd but I'm sure you have good reasons!
      - DRMacIver 12y ago
        Yeah, the google ranking of Hypothesis when you search for python quickcheck is a thing that's been bugging me. There's a blog post from 2013 about it on the front page but none of the various places the project exists appear. I'm not sure what to do about it. I'm planning to do a bunch of shopping around and promoting of the library when the 1.0 release happpens (probably next week), so hopefully that should help. The CLA is for a mix of weird personal reasons around open source and cynically practical ones. Hypothesis is 100% always going to remain free, but I'm trying to figure out ways that I can make money out of working on it (I've put a ton of work into it, so it would be a shame not to) and that's a lot easier if I retain the copyright. I may decide it's not useful and drop it later, but it's a lot easier to have it now and drop it later than it is to try and retroactively get one if it turns out I need it.
    - sa2ajj 12y ago
      Thank you for bringing it in: I've been looking for a pure Python analogue of QuickCheck.
toolslive 12y ago
I have seen people do the same with their C code: code it up in C, expose it using the ffi and use quickcheck to exercise it. It's not only Haskell that can be used this way. OCaml + ctypes is also an option.
- sa2ajj 12y ago
  I remember on one of the Erlang User Conferences there was an [impressive] presentation of testing C code with an erlang "driver".
bensummers 12y ago
You might also want to have a look at hypothesis: https://github.com/DRMacIver/hypothesis https://github.com/DRMacIver/hypothesis http://hypothesis.readthedocs.org/en/master/ http://hypothesis.readthedocs.org/en/master/ It's a QuickCheck inspired python testing library with some interesting new ideas.
cpa 12y ago
Neat, but there's a subtelty: at the end of the article, we only proved that square is error-free[0] when given an Int as input (because the haskell version of square is Int -> IO Int). But Python being loosely typed, I could call square on a float, and in this case nothing can be said of the python function. Also, you have to make sure haskell's Ints and Python's Int are the same on your platform (one could be 32 bits and the other 64 bits). [0] we did not prove anything, but just ran a bunch of tests, but you get the point.
- Ded7xSEoPKYNsDd 12y ago
  > Also, you have to make sure haskell's Ints and Python's Int are the same on your platform (one could be 32 bits and the other 64 bits). Aren't they both infinite-resolution?
  - DRMacIver 12y ago
    No, the infinite precision integer type in Haskell is called Integer. Int is a platform dependent fixed size int.
codelion 12y ago
Neat trick using the FFI for this. Also, quickCheck like libraries are available in other languages e.g JUnit-QuickCheck for Java - https://blog.sourceclear.com/property-based-testing-for-java/ https://blog.sourceclear.com/property-based-testing-for-java...

43 points by Russell91 12y ago

Or of course, you can skip haskell and just use the python port https://pypi.python.org/pypi/pytest-quickcheck https://pypi.python.org/pypi/pytest-quickcheck

sa2ajj 12y ago
It seems to provide random data generation only. I'd say _the_ feature of QuickCheck is minimisation of the counter example.
- DRMacIver 12y ago
  Yep. This is why I wrote Hypothesis: http://hypothesis.readthedocs.org/ http://hypothesis.readthedocs.org/ / https://pypi.python.org/pypi/hypothesis https://pypi.python.org/pypi/hypothesis It's a lot more fully featured than the alternatives. It still lacks a few features of quickcheck (coarbitrary for function generation, labelling of examples), but it does example minimization and a whole bunch of things even quickcheck doesn't.
  - DRMacIver 12y ago
    Realised I should have linked to http://hypothesis.readthedocs.org/en/master/internals.html http://hypothesis.readthedocs.org/en/master/internals.html to back up that "a whole bunch of things even quickcheck doesn't" claim. :-)
  - deleted 12y ago
    [deleted]
  - teh 12y ago
    Hypothesis looks fantastic. A friend of mine and me spent a while on Google collecting Python QC implementations but didn't find any that implemented shrinking. Hypothesis escaped our search. I don't know how but the project could do with better Google ranking :) It has some really useful features like storing the counterexamples. The CLA is a bit odd but I'm sure you have good reasons!
    - DRMacIver 12y ago
      Yeah, the google ranking of Hypothesis when you search for python quickcheck is a thing that's been bugging me. There's a blog post from 2013 about it on the front page but none of the various places the project exists appear. I'm not sure what to do about it. I'm planning to do a bunch of shopping around and promoting of the library when the 1.0 release happpens (probably next week), so hopefully that should help. The CLA is for a mix of weird personal reasons around open source and cynically practical ones. Hypothesis is 100% always going to remain free, but I'm trying to figure out ways that I can make money out of working on it (I've put a ton of work into it, so it would be a shame not to) and that's a lot easier if I retain the copyright. I may decide it's not useful and drop it later, but it's a lot easier to have it now and drop it later than it is to try and retroactively get one if it turns out I need it.
  - sa2ajj 12y ago
    Thank you for bringing it in: I've been looking for a pure Python analogue of QuickCheck.

toolslive 12y ago

I have seen people do the same with their C code: code it up in C, expose it using the ffi and use quickcheck to exercise it. It's not only Haskell that can be used this way. OCaml + ctypes is also an option.

sa2ajj 12y ago
I remember on one of the Erlang User Conferences there was an [impressive] presentation of testing C code with an erlang "driver".

bensummers 12y ago

You might also want to have a look at hypothesis: https://github.com/DRMacIver/hypothesis https://github.com/DRMacIver/hypothesis http://hypothesis.readthedocs.org/en/master/ http://hypothesis.readthedocs.org/en/master/ It's a QuickCheck inspired python testing library with some interesting new ideas.

cpa 12y ago

Neat, but there's a subtelty: at the end of the article, we only proved that square is error-free[0] when given an Int as input (because the haskell version of square is Int -> IO Int). But Python being loosely typed, I could call square on a float, and in this case nothing can be said of the python function. Also, you have to make sure haskell's Ints and Python's Int are the same on your platform (one could be 32 bits and the other 64 bits). [0] we did not prove anything, but just ran a bunch of tests, but you get the point.

Ded7xSEoPKYNsDd 12y ago
> Also, you have to make sure haskell's Ints and Python's Int are the same on your platform (one could be 32 bits and the other 64 bits). Aren't they both infinite-resolution?
- DRMacIver 12y ago
  No, the infinite precision integer type in Haskell is called Integer. Int is a platform dependent fixed size int.

codelion 12y ago

Neat trick using the FFI for this. Also, quickCheck like libraries are available in other languages e.g JUnit-QuickCheck for Java - https://blog.sourceclear.com/property-based-testing-for-java/ https://blog.sourceclear.com/property-based-testing-for-java...