3 ms·
>Now, suppose that the product manager comes to us one day and says, "We have a new design, we'd now like it to say "Greetings Brandon, welcome to our website!"
by MoreQARespect 3y ago
>Now, suppose that the product manager comes to us one day and says, "We have a new design, we'd now like it to say "Greetings Brandon, welcome to our website!"
I have been writing YAML based tests which rewrite themselves based upon program output in these cases.
Change the code -> run one script and now the test is updated. There is near zero test maintenance cost for surface level changes like this message or a tweaked JSON output from an API.
I call it snapshot test driven development.
I agree with the overall sentiment - testing ought to considered an investment that provides a recognized ROI.
- brundolf 3y agoThere's a similar concept of "snapshots" in UI testing, which I maybe should have included in the post I think snapshots definitely lower (but don't eliminate) the cost of a certain class of tests, and can sometimes tip the scales from "not worth it" to "worth it"
- MoreQARespect 3y agoIt's almost the same thing, I think. I do the same thing with text artefacts and screenshots. There is a hidden cost to snapshot testing - theyre highly susceptible to flakiness. The tests have to be hermetic and the all the code has to be deterministic or it doesnt work. Achieving those two things is not always easy.
- brundolf 3y agoYeah. They're highly flaky, but counterbalanced by being very easy to update when they break IMO, if you've decided to test flaky behaviors of the code anyway, you may as well do it in a way where updating is really easy