3 ms·

"Automated testing only shows its real value when you go back to change code that was working before" end to end tests, integration tests, regression tests, et

by shados 10y ago

"Automated testing only shows its real value when you go back to change code that was working before" end to end tests, integration tests, regression tests, etc, yeah. Unit tests though...usually no. Often if you're making any kind of significant change, the entire code paths may get refactored away or change too much and the test will get nuked anyway. And it's that kind of test that usually confuses people, so it's worth understanding. A unit test's goals are many: It proves at authoring time that the code works. It saves you the time right away of having to go through the UI or spinning up a server just to validate a function is working. It proves that you thought about specific edge cases (and if you do testing consistently, the lack of test is your evidence of unconsidered edge cases. It's documentation of all of the things you considered when writing the code. It is an example of how to use the code with all it's use case. And when you nuke a piece of code away, the failing tests are now a guide of all the cases you have to make sure are truly no longer necessary. If, in the future, you do a refactor of an implementation detail (so the existing tests are still valid), then that's bonus as you get green/red validation. But in practice, that is less common than all of the other reasons for testing. That's why the "I don't expect this to change" thing isn't really a reason for or against writing tests.

laserDinosaur 10y ago
>Unit tests though...usually no. Often if you're making any kind of significant change, the entire code paths may get refactored away or change too much and the test will get nuked anyway. That pretty accurately describes my experience with unit tests. I've been part of several projects where we had pretty comprehensive unit tests (I'd say small to medium sized projects) and I never managed to get as much use out of unit tests as I liked. After one or two big refactors most of the tests needed to be, as you said, nuked anyway. While seeing all the pretty green lights is reassuring, they are rarely working when you most need them - during large refactors which blows away big sections of code. I picture unit tests as a row of black boxes sitting on a table in certain positions. Unit tests are great when you don't move the black boxes but do change the mysterious processes are running inside them. But refactoring is rarely ever that isolated in programming since you tend to move some of the black boxes around, remove some entirely, add some new ones, change the contents. To then expect the unit tests to give you back useful information on whats broken is rarely possible. I've had more success with e2e testing using things like Selenium, but it's still frustrating as a developer to read articles about how great unit testing is (like this root comment) and never able to actually get a decent working version of it in your projects (because of the reasons mentioned by this parent comment).
Jach 10y ago
The biggest benefit I've been seeing from unit tests is in writing more of them inside a code base sorely lacking them. In that case there's a big side-effect of making the underlying code unit-testable (that is, if you don't totally cheat every time with something like PowerMock). Some tests may get blown away in refactoring, but testing the new code has the same benefit. In many cases this makes the implementation uglier but doesn't hurt its clarity much (oftentimes it can improve it), especially with IDE assistance. Building up small oases of dumber/more verbose code that has unit tests seems like the best way to wrangle legacy code into something that anyone else on the team can understand and not mess up 6 months from now when it's their turn to have to touch it for the first time. Of course no one wants to touch that 200-line, 10-levels-of-indentation monster method, but bringing just a little bit more and more of it under unit test over time will help a lot. Every other benefit of unit testing that you listed besides the time saving aspect (e.g. why launch a big e2e test if you can test the same thing in an xunit-like context? even if it's not strictly a "unit" test) pales in comparison of the benefit of making crap code nicer to work with. A corollary is that if your code is already nice to work with, and you have a system to keep it that way, unit tests won't be very valuable. (Though other tests, which may or may not be in an xunit-like context, may still be quite valuable.)

by shados 10y ago

laserDinosaur 10y ago

>Unit tests though...usually no. Often if you're making any kind of significant change, the entire code paths may get refactored away or change too much and the test will get nuked anyway. That pretty accurately describes my experience with unit tests. I've been part of several projects where we had pretty comprehensive unit tests (I'd say small to medium sized projects) and I never managed to get as much use out of unit tests as I liked. After one or two big refactors most of the tests needed to be, as you said, nuked anyway. While seeing all the pretty green lights is reassuring, they are rarely working when you most need them - during large refactors which blows away big sections of code. I picture unit tests as a row of black boxes sitting on a table in certain positions. Unit tests are great when you don't move the black boxes but do change the mysterious processes are running inside them. But refactoring is rarely ever that isolated in programming since you tend to move some of the black boxes around, remove some entirely, add some new ones, change the contents. To then expect the unit tests to give you back useful information on whats broken is rarely possible. I've had more success with e2e testing using things like Selenium, but it's still frustrating as a developer to read articles about how great unit testing is (like this root comment) and never able to actually get a decent working version of it in your projects (because of the reasons mentioned by this parent comment).

Jach 10y ago

The biggest benefit I've been seeing from unit tests is in writing more of them inside a code base sorely lacking them. In that case there's a big side-effect of making the underlying code unit-testable (that is, if you don't totally cheat every time with something like PowerMock). Some tests may get blown away in refactoring, but testing the new code has the same benefit. In many cases this makes the implementation uglier but doesn't hurt its clarity much (oftentimes it can improve it), especially with IDE assistance. Building up small oases of dumber/more verbose code that has unit tests seems like the best way to wrangle legacy code into something that anyone else on the team can understand and not mess up 6 months from now when it's their turn to have to touch it for the first time. Of course no one wants to touch that 200-line, 10-levels-of-indentation monster method, but bringing just a little bit more and more of it under unit test over time will help a lot. Every other benefit of unit testing that you listed besides the time saving aspect (e.g. why launch a big e2e test if you can test the same thing in an xunit-like context? even if it's not strictly a "unit" test) pales in comparison of the benefit of making crap code nicer to work with. A corollary is that if your code is already nice to work with, and you have a system to keep it that way, unit tests won't be very valuable. (Though other tests, which may or may not be in an xunit-like context, may still be quite valuable.)