5 ms·

I attribute some of the greatest successes of my career in part to having good unit test coverage. I have seen no other pragmatic way to solve the problem that,

I attribute some of the greatest successes of my career in part to having good unit test coverage. I have seen no other pragmatic way to solve the problem that, in the worst case, a minor software update can be O(N) expensive to make, where N is the size or complexity of your program. That little update has the habit of breaking things in code paths you'd never expect. With a good suite of unit tests, you can validate that your program likely still functions, and, far more importantly, you can do the validation quickly, which results fast feedback loops and iteration times. Fast feedback loops are essential to high developer productivity. When developers are being slow, in my experience it usually is related to the system they are working on having slow feedback cycles. > The basic "problem" I have with tests is that - at least maybe in my experience - things tend to go wrong in cases that you haven't thought about so your tests don't cover them anyway. You are right in that unit testing can't make up for bad engineering. One advantage of investing time in it, though, is it gives you potentially more opportunity to think through those cases. And, as a consolation, prize, you write unit tests for bugs that make it through and now you have a great regression suite. Furthermore, a major issue that often comes when unit tests are omitted or not taken seriously, is that these tests can't easily be added after the fact, A system has to be designed with unit testing in mind, or else you end up with a bunch of untestable code. I believe that many legacy codebases remain without tests because the act of refactoring them enough to support any testing at all would be both time-consuming and high risk in terms of potential breakage.

trgn 4y ago
I agree 100%, but just want to add my2c. Unit tests are in general never wasted, in the sense that they trap regressions before release. I don't think they necessarily are good to validate work though, or at least, they do so poorly. It's easy to write lots of tests which don't cover the areas your program will bork on in production. Which I think is point of GP. Value of tests compound over time. The longer the software is in production and maintained, the more work the unit tests have been doing. It could be a paltry 20% code coverage set of tests, but nonetheless they're fighting the good fight. The value of a test suite is expressed in the integral over time. Few tests can do a lot of work; on the flip side, a lot of tests are useless when they're continuously thrown away due to pivots in business logic.
- PathOfEclipse 4y ago
  > I don't think they necessarily are good to validate work though, or at least, they do so poorly. For me, I've found they actually do help significantly to catch bugs even for my initial PRs. To know your code works, you either have to run it manually or else write a unit test, and you must do so for all relevant code paths. How much time do you save by avoiding the unit test, if any at all, and how long before you recoup that time via your integral? In some projects, I think the time saved is closed to zero, whereas the benefits accrue almost immediately. You are right that there is a tradeoff here, and I have been part of a project where I wrote a bunch of tests for code that ended up being a throwaway prototype. You could argue that I wasted time and resources there. On the other hand, I've seen more prototypes get shipped to production than scrapped, so maybe the calculated risk is worth it even in scrappier contexts.

trgn 4y ago

I agree 100%, but just want to add my2c. Unit tests are in general never wasted, in the sense that they trap regressions before release. I don't think they necessarily are good to validate work though, or at least, they do so poorly. It's easy to write lots of tests which don't cover the areas your program will bork on in production. Which I think is point of GP. Value of tests compound over time. The longer the software is in production and maintained, the more work the unit tests have been doing. It could be a paltry 20% code coverage set of tests, but nonetheless they're fighting the good fight. The value of a test suite is expressed in the integral over time. Few tests can do a lot of work; on the flip side, a lot of tests are useless when they're continuously thrown away due to pivots in business logic.

PathOfEclipse 4y ago
> I don't think they necessarily are good to validate work though, or at least, they do so poorly. For me, I've found they actually do help significantly to catch bugs even for my initial PRs. To know your code works, you either have to run it manually or else write a unit test, and you must do so for all relevant code paths. How much time do you save by avoiding the unit test, if any at all, and how long before you recoup that time via your integral? In some projects, I think the time saved is closed to zero, whereas the benefits accrue almost immediately. You are right that there is a tradeoff here, and I have been part of a project where I wrote a bunch of tests for code that ended up being a throwaway prototype. You could argue that I wasted time and resources there. On the other hand, I've seen more prototypes get shipped to production than scrapped, so maybe the calculated risk is worth it even in scrappier contexts.