4 ms·

It's standard practice in the frontend world https://jestjs.io/docs/snapshot-testing https://jestjs.io/docs/snapshot-testing

sensanaty 2y ago
In my experience snapshots have largely fallen out of vogue, at least as far as them blocking deploys goes. They're usually flaky, change often, are slow to test and it's way too easy to have false positives which lulls people into complacency and just regenerating the snapshots as soon as they reach a failure. I'm guilty of that myself, unless I can spot the breakage immediately I pretty much just regen the snapshot by default subconsciously because of how often I've had to do that for false positives
- yawaramin 2y ago
  Then what's to stop you from making any unit test pass by just changing the expectations? The best testing technique in the world can't save us from developer error.
  - Hasu 2y ago
    The problem with snapshot tests is that they tend to fail when something is changed. One property of good tests is that they only fail when something is broken. Snapshot tests aren't really automated tests, because you have to manually check the outputs to see if failures are genuine. It's a reminder to do manual checking but it's still not an automated process. > The best testing technique in the world can't save us from developer error. Sure, but using a testing technique that increases developer error is unwise, and snapshot testing has done that every time I've seen it used, so I don't use it anymore.
    - yawaramin 2y ago
      > they tend to fail when something is changed. Then fix the test so that it fails only when something breaks? Do people not fix flaky, overly broad, or incorrect unit tests? How is a snapshot test any different?
      - DecoySalamander 2y ago
        A fix here would be to drop snapshot testing altogether, since being flaky and overly broad is a natural result of dumb diffing of your app's output.
      - Hasu 2y ago
        I'm not sure you understand what snapshot testing is. It's a form of testing where you render frontend components and take a "snapshot", storing the output and saving it for later, then on the next test run, the component is rendered again and compared to the saved snapshot. Any change that modifies the rendering of any component under test will break snapshot tests. It's literally just a test that says, "Is this render function still the same as it used to be?"
        yawaramin 2y ago
        I'm not sure you understand how to capture snapshots at the correct level of granularity. If you have snapshots that are breaking for irrelevant changes, then by definition those changes don't need to be snapshotted, do they? Cut down the snapshot to only the relevant part of the rendered HTML so that when it fails, you know something broke. Eg in htmx we can add a snapshot test for the pagination component I mentioned in this comment: https://news.ycombinator.com/item?id=42619553 https://news.ycombinator.com/item?id=42619553 This snapshot will break if I have a bug in my pagination logic implementation, and it's highly unlikely to fail for random irrelevant changes.
        Hasu 2y ago
        It would still break if you need to modify that output in any way. Need to add a class for styling? The snapshot breaks. Need to add a new hx-attribute? The snapshot breaks. It's not tied to the logic, it's tied to whatever markup you output. You've reduced the surface area of the problem, but not eliminated it. > If you have snapshots that are breaking for irrelevant changes, then by definition those changes don't need to be snapshotted, do they? You're so close to getting it.

sensanaty 2y ago

In my experience snapshots have largely fallen out of vogue, at least as far as them blocking deploys goes. They're usually flaky, change often, are slow to test and it's way too easy to have false positives which lulls people into complacency and just regenerating the snapshots as soon as they reach a failure. I'm guilty of that myself, unless I can spot the breakage immediately I pretty much just regen the snapshot by default subconsciously because of how often I've had to do that for false positives

yawaramin 2y ago
Then what's to stop you from making any unit test pass by just changing the expectations? The best testing technique in the world can't save us from developer error.
- Hasu 2y ago
  The problem with snapshot tests is that they tend to fail when something is changed. One property of good tests is that they only fail when something is broken. Snapshot tests aren't really automated tests, because you have to manually check the outputs to see if failures are genuine. It's a reminder to do manual checking but it's still not an automated process. > The best testing technique in the world can't save us from developer error. Sure, but using a testing technique that increases developer error is unwise, and snapshot testing has done that every time I've seen it used, so I don't use it anymore.
  - yawaramin 2y ago
    > they tend to fail when something is changed. Then fix the test so that it fails only when something breaks? Do people not fix flaky, overly broad, or incorrect unit tests? How is a snapshot test any different?
    - DecoySalamander 2y ago
      A fix here would be to drop snapshot testing altogether, since being flaky and overly broad is a natural result of dumb diffing of your app's output.
    - Hasu 2y ago
      I'm not sure you understand what snapshot testing is. It's a form of testing where you render frontend components and take a "snapshot", storing the output and saving it for later, then on the next test run, the component is rendered again and compared to the saved snapshot. Any change that modifies the rendering of any component under test will break snapshot tests. It's literally just a test that says, "Is this render function still the same as it used to be?"
      - yawaramin 2y ago
        I'm not sure you understand how to capture snapshots at the correct level of granularity. If you have snapshots that are breaking for irrelevant changes, then by definition those changes don't need to be snapshotted, do they? Cut down the snapshot to only the relevant part of the rendered HTML so that when it fails, you know something broke. Eg in htmx we can add a snapshot test for the pagination component I mentioned in this comment: https://news.ycombinator.com/item?id=42619553 https://news.ycombinator.com/item?id=42619553 This snapshot will break if I have a bug in my pagination logic implementation, and it's highly unlikely to fail for random irrelevant changes.
        Hasu 2y ago
        It would still break if you need to modify that output in any way. Need to add a class for styling? The snapshot breaks. Need to add a new hx-attribute? The snapshot breaks. It's not tied to the logic, it's tied to whatever markup you output. You've reduced the surface area of the problem, but not eliminated it. > If you have snapshots that are breaking for irrelevant changes, then by definition those changes don't need to be snapshotted, do they? You're so close to getting it.