4 ms·
Testing is an infinitely complicated subject involving very difficult tradeoffs, but there are two axes I like to break down on when thinking about it: 1) Cove
by carsongross 15y ago
Testing is an infinitely complicated subject involving very difficult tradeoffs, but there are two axes I like to break down on when thinking about it:
1) Coverage vs. Speed/Understandability: you want as much coverage as is compatible with relatively fast feedback and a suite where, when something goes wrong, it is sufficiently obvious what went wrong. As a system increases (non-linearly) in complexity, this becomes a more and more difficult tradeoff and you should expect to feel increasingly bad about the state of affairs as your code base grows in size: either stuff is gonna be slow and incomprehensible or you are not gonna have the coverage you want. Probably both.
2) High-level vs. Unit tests: as your system matures, the high level tests are going to be the most valuable over time, since they are the abstract specification of the behavior of the system. They are also going to be the hardest to understand when they break, since they are inherently abstract. Your unit tests will keep running quickly and breaking in obvious ways, but will become increasingly useless for verifying system validity as second-order effects start dominating your bugs, and they will introduce lots of friction if you are trying to refactor code (which might be good: many a refactor has gone bad in my career.)
So, as usual, I come to a pessimistic conclusion: tests are necessary and, as your code base increases in size and complexity, they will come to dominate your development process. Holding test quality constant, the high level tests will run the slowest and break in the most incomprehensible manner, but will be the most valuable test you have, and you will develop a love-hate relationship with them.
So, my take aways are:
1) Write unit tests, but be prepared to nuke them once your implementation is done in favor of higher level tests. Make sure they run very fast so they don't get in the way of your development.
2) Try to write very clean high-level tests. This is hard and you have to trade off how much time you put into it vs. other priorities. Strive to make these fast, but accept that they will be slower than you'd like and will cause you a lot of pain. Every once in a while, they are gonna save you.
- charlescharles 15y agoHigh-level tests and unit tests go hand in hand. Ideally you are striving for 100% coverage for both your high level tests and your unit tests. The value of that is that when a high-level test fails in some inscrutable way, in theory a corresponding unit test should also fail, which should make the actually point of breakage clear.