3 ms·
I think one mental model for tests is that they're simply another kind of automation. The fundamental question of automation is: will I repeat this task enough
by munificent 8y ago
I think one mental model for tests is that they're simply another kind of automation.
The fundamental question of automation is: will I repeat this task enough such that the amortized saving outweighs the cost of writing a script to do it?
Whenever you're thinking of adding a test X, quickly consider how often you and your team are likely to need to manually test X if you don't. Also factor in the cost of writing test X (though that's tricky because sometimes you need to build test architecture Y, which lowers the costs of writing many tests...).
If it's a piece of code that's buggy, changing frequently, brittle, or important, then you're likely to need to validate its behavior again and again. It's probably worth writing a unit test for it.
If it's an end user experience that involves a lot of services maintained by different teams and tends to fall apart often, it's probably worth writing an integration test instead of having to keep manually testing every time the app goes down.
If it's an API with lots of important users using it for all sorts of weird edge cases and you don't want to have to manually repro each of the existing weird edge cases any time you add a new one, it's probably worth writing some black box tests for it.
But if it's code that's simple, boring, stable, short-lived, or unimportant, your time may be better spent elsewhere.
Another model is that your tests represent the codified understanding of what the system is. This can be very helpful if you have a lot of team churn. How do you make sure a new person doesn't break something when "something" isn't well-defined somewhere? Tests are a great way to pin that down. If this is a priority, then it becomes more important to write tests for things where the test actually very rarely fails. It's more executable system specification than it is automation.
- chriswarbo 8y agoI agree with the idea that we're mostly automating manual work. The really nice thing about automated tests is that they can also go much further than I would ever bother to manually. For example, I might manually check that a couple of product pages are showing, with an image, price and description. With automated testing I can check all product pages, every time. It can also test a tedious amount of interactions, e.g. various interleavings of "add to cart", "remove from cart", "checkout", "press back button", etc. Manual testing still gives me confidence that automated tests can't, e.g. that an "unknown unknown" hasn't taken things down. Yet that's all I need to check manually; if pages are loading and look right, I don't feel a need to manually check all of the ecommerce arithmetic, permission logic, etc.