4 ms·

Snapshot testing is great, and I wish more test frameworks included first-class support for them. This means that they can auto update with a flag, and can be s

Snapshot testing is great, and I wish more test frameworks included first-class support for them. This means that they can auto update with a flag, and can be stored either in the source inline or in an external file (both modes have different use cases). Note that doc tests can also be a form of this, e.g. in Python's. "Expect tests" seems like a bad name, since that covers all tests.

chii 4y ago
i find that snapshot testing gets overused in javascript - and mistakes can creep in easily, and if the snapshot is big, and in a separate file, code review can miss it. I much prefer property based testing over expectation based testing. You have to explicitly think about what properties hold true about the thing you're writing. For example, fib(N+1) = fib(N) + fib(N), so this property can be tested for all N; primitive generators can easily generate the data, and good composition framework can easily generate complex data from primitive data. Of course, you have to have a property you can specify easily. Otherwise, it'd be exactly the same as expectation based testing.
- IanCal 4y ago
  Every single time I've introduced property based testing, even as a simple example, I've discovered a bug in either the code or the spec. I've found a bug in a Haskell program about fib generation - your test would work (if fixed for the subtractions) but incorrectly as there was an overflow in the addition. A basic property of "fib(n+1) > fib(n)" for n>1 finds this. I like this type of testing as it asks you to more generally consider what guarantees your code is making about its operation. Edit - your example is a good one and necessary, I just wanted to add a bit extra as I really like property based testing
- sesm 4y ago
  Snapshot testing works well for component systems, especially with storybook. There is a service called Chromatic that lets you diff component changes visually using storybook output.
tantalor 4y ago
> update with a flag Yes this is right level of automation, not whatever this article is going on about with the editor integration. Yuck.
- thedufer 4y ago
  The open source use pattern for expect tests in OCaml (via dune) is exactly as you describe (see https://dune.readthedocs.io/en/stable/tests.html https://dune.readthedocs.io/en/stable/tests.html) - you run the tests with `--auto-promote` to tell it to update. The editor integration is a very simple keybinding on top of more generic tooling.

chii 4y ago

i find that snapshot testing gets overused in javascript - and mistakes can creep in easily, and if the snapshot is big, and in a separate file, code review can miss it. I much prefer property based testing over expectation based testing. You have to explicitly think about what properties hold true about the thing you're writing. For example, fib(N+1) = fib(N) + fib(N), so this property can be tested for all N; primitive generators can easily generate the data, and good composition framework can easily generate complex data from primitive data. Of course, you have to have a property you can specify easily. Otherwise, it'd be exactly the same as expectation based testing.

IanCal 4y ago
Every single time I've introduced property based testing, even as a simple example, I've discovered a bug in either the code or the spec. I've found a bug in a Haskell program about fib generation - your test would work (if fixed for the subtractions) but incorrectly as there was an overflow in the addition. A basic property of "fib(n+1) > fib(n)" for n>1 finds this. I like this type of testing as it asks you to more generally consider what guarantees your code is making about its operation. Edit - your example is a good one and necessary, I just wanted to add a bit extra as I really like property based testing
sesm 4y ago
Snapshot testing works well for component systems, especially with storybook. There is a service called Chromatic that lets you diff component changes visually using storybook output.

tantalor 4y ago

> update with a flag Yes this is right level of automation, not whatever this article is going on about with the editor integration. Yuck.

thedufer 4y ago
The open source use pattern for expect tests in OCaml (via dune) is exactly as you describe (see https://dune.readthedocs.io/en/stable/tests.html https://dune.readthedocs.io/en/stable/tests.html) - you run the tests with `--auto-promote` to tell it to update. The editor integration is a very simple keybinding on top of more generic tooling.