3 ms·

This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each funct

This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.

rcap 7y ago
This is basically what they call snapshot testing
noobiemcfoob 7y ago
I really like this idea. So, broken out, would the development process look like: 1) Write code 2) Open interpreter (ipython) 3) Import code and call it manually 4) This magic library will have observed and created a testcase module from your execution If the library can use the observed and recorded testcase to extrapolate far beyond your manual call, this could be a really interesting way of testing python code.
- ACow_Adonis 7y ago
  This is actually the idea behind a little library I developed for my own use in common lisp called testy. Because the common lisp environment already defines special variables that contain the last form evaluated and the return value of the last form evaluated, when you evaluate code at the REPL and it succeeds, you call the (testy:test) function, which captures them and wraps them up as a test for the package you're working on, adding it to the database of tests for the package. Not only was it useful for development (no need to specify your tests up front, you write your code until it works, then that state "is" the test), but it was also helpful for fixing bugs that you hadn't thought of up front (you fix the bug and then that state is captured as a new test as well).
ryall 7y ago
I remember finding something like this a few years ago for testing rest apis. It was very cool but for the life of me I can't recall the name :(
- Izkata 7y ago
  Possibly vcr [0], except that's for mocking out APIs in unit tests. There's a whole bunch of ports to other languages midway down the page. [0] https://github.com/vcr/vcr https://github.com/vcr/vcr
OJFord 7y ago
Instagram created 'MonkeyType' to do something similar for type annotations. A lower barrier way of trying this out could be to run MonkeyType in combination with OP's hypothesis-auto (which uses type annotations to generate the tests).
- westurner 7y ago
  pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram) [3] all do dynamic / runtime PEP-484 type annotation type inference [4] [1] https://github.com/google/pytype https://github.com/google/pytype [2] https://github.com/dropbox/pyannotate https://github.com/dropbox/pyannotate [3] https://github.com/Instagram/MonkeyType https://github.com/Instagram/MonkeyType [4] https://news.ycombinator.com/item?id=19454411 https://news.ycombinator.com/item?id=19454411
knubie 7y ago
I came across something similar for Clojure recently. https://github.com/ahungry/determinism/ https://github.com/ahungry/determinism/
timothycrosley 7y ago
I'd use it!
anonytrary 7y ago
This sounds like snapshot testing. It is the responsibility of the person to browse the produced snapshots to ensure they are correct.

rcap 7y ago

This is basically what they call snapshot testing

noobiemcfoob 7y ago

I really like this idea. So, broken out, would the development process look like: 1) Write code 2) Open interpreter (ipython) 3) Import code and call it manually 4) This magic library will have observed and created a testcase module from your execution If the library can use the observed and recorded testcase to extrapolate far beyond your manual call, this could be a really interesting way of testing python code.

ACow_Adonis 7y ago
This is actually the idea behind a little library I developed for my own use in common lisp called testy. Because the common lisp environment already defines special variables that contain the last form evaluated and the return value of the last form evaluated, when you evaluate code at the REPL and it succeeds, you call the (testy:test) function, which captures them and wraps them up as a test for the package you're working on, adding it to the database of tests for the package. Not only was it useful for development (no need to specify your tests up front, you write your code until it works, then that state "is" the test), but it was also helpful for fixing bugs that you hadn't thought of up front (you fix the bug and then that state is captured as a new test as well).

ryall 7y ago

I remember finding something like this a few years ago for testing rest apis. It was very cool but for the life of me I can't recall the name :(

Izkata 7y ago
Possibly vcr [0], except that's for mocking out APIs in unit tests. There's a whole bunch of ports to other languages midway down the page. [0] https://github.com/vcr/vcr https://github.com/vcr/vcr

OJFord 7y ago

Instagram created 'MonkeyType' to do something similar for type annotations. A lower barrier way of trying this out could be to run MonkeyType in combination with OP's hypothesis-auto (which uses type annotations to generate the tests).

westurner 7y ago
pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram) [3] all do dynamic / runtime PEP-484 type annotation type inference [4] [1] https://github.com/google/pytype https://github.com/google/pytype [2] https://github.com/dropbox/pyannotate https://github.com/dropbox/pyannotate [3] https://github.com/Instagram/MonkeyType https://github.com/Instagram/MonkeyType [4] https://news.ycombinator.com/item?id=19454411 https://news.ycombinator.com/item?id=19454411

knubie 7y ago

I came across something similar for Clojure recently. https://github.com/ahungry/determinism/ https://github.com/ahungry/determinism/

timothycrosley 7y ago

I'd use it!

anonytrary 7y ago

This sounds like snapshot testing. It is the responsibility of the person to browse the produced snapshots to ensure they are correct.