5 ms·

Is it really? Most unit tests I've encountered are garbage. They only test for bugs that are incredibly unlikely to happen. Integration tests and fuzz testing

by NavinF 5y ago

Is it really? Most unit tests I've encountered are garbage. They only test for bugs that are incredibly unlikely to happen. Integration tests and fuzz testing is where it's at. Anyone can add or improve an integration test within an hour and find several bugs that real users would encounter.

ternaryoperator 5y ago
All that could be true and it still would not suggest that no unit tests at all is a valid choice.
Jtsummers 5y ago
> They only test for bugs that are incredibly unlikely to happen. Well, that might be the problem. Testing "for bugs" isn't a terribly helpful activity. Testing for behavior is much more helpful, and unit tests can definitely be applied to that aspect of the system. Of course, it requires you to properly decompose the system into subsystems (modules) with loose coupling that can be individually (or in small sets) loaded into test harnesses for testing.
ddlutz 5y ago
Maybe you are not working on the right projects or with the right people, because good unit tests are extremely valuable. At a minimum, ensuring that making a change to some existing code doesn't break existing behavior is valuable.
titzer 5y ago
Strong disagree. Effective unit tests accelerate development by allowing fast turnaround and refactoring with confidence. There are bad unit tests, of course, that are highly coupled with the code that they test. I have written tens of thousands of unit tests in my career, and by far the most useful ones are the ones that point you to the exact subsystem in which a bug you've just introduced during refactoring/re-implementing lies. Bad unit tests are brittle "change detectors" and make you constantly adjust them for no real reason. I find mocks and interaction-based tests (rather than simple input/output tests) are this way. I flat-out do not use mocks for this very reason. They are intricate, involved, and take huge amounts of time to load context when they fail. Don't write such tests. If your unit tests are garbage, it's because they were not thought out very much and commit sins.
- bcrosby95 5y ago
  > I flat-out do not use mocks for this very reason. They are intricate, involved, and take huge amounts of time to load context when they fail. Don't write such tests. This is why I prefer integration tests. Most of my projects interact heavily with a database. Making it an integration test unless I mock it out. Which I don't do, because in my experience, doing so tends to hide lots of bugs.
  - titzer 5y ago
    > This is why I prefer integration tests. IMO integration tests are great for making sure that your system actually functions with all its parts together. Nothing but an integration test can do that. But unit tests serve a different role: reducing the scope of the system under test so that you can pinpoint exactly what part failed when it does fail. They help you iterate quickly in refactoring, and when something goes wrong in production (or a bug report from the field), they narrow the cone of uncertainty by eliminating from consideration the parts of the system that are working correctly. In my experience, this rapidly cuts down on the amount of wild goose chases one goes on and overall makes a team more efficient. Then once a bug in the wild is found, the system responsible for the failure gets new regression tests and unit tests. Sometimes, I wrote unit tests for units I am suspicious of as I am debugging, and even if those tests don't find a bug, I might keep them. I worked for 6+ years on V8 and the parts of the system that were the least unit tested seemed to be home to far more production issues than others. It also helps a lot to keep your unit tests running super fast. Under 1 second is ideal.
- deleted 5y ago
  [deleted]
- thom 5y ago
  I think I agree with this, although I also think a lot of people would still call this integration testing. Basically a lot of code units are implementation details and you want to be free to refactor them fearlessly without having to write double the code each time tightly unit testing each part. Much better to narrow your system to a few well defined boundaries that deliver some clear value. The code behind that interface can be as deep as you like but you don’t really need to push tests down into the depths. As I say, some people will still think of this as integration testing just because there’s a lot of moving parts working together. But often you’re just testing fairly pure code with no external or IO dependencies so I’m happy thinking of these as unit tests still. Where those types of dependencies exist I do still mock though.
Rd6n6 5y ago
Except for unusually complex logic, almost every bug I’ve ever seen has been configuration or integration related. I do unit tests in safety critical software, but if I actually need to catch a bug I usually worry more about integration or end to end tests

by NavinF 5y ago

ternaryoperator 5y ago

All that could be true and it still would not suggest that no unit tests at all is a valid choice.

Jtsummers 5y ago

> They only test for bugs that are incredibly unlikely to happen. Well, that might be the problem. Testing "for bugs" isn't a terribly helpful activity. Testing for behavior is much more helpful, and unit tests can definitely be applied to that aspect of the system. Of course, it requires you to properly decompose the system into subsystems (modules) with loose coupling that can be individually (or in small sets) loaded into test harnesses for testing.

ddlutz 5y ago

Maybe you are not working on the right projects or with the right people, because good unit tests are extremely valuable. At a minimum, ensuring that making a change to some existing code doesn't break existing behavior is valuable.

titzer 5y ago

Strong disagree. Effective unit tests accelerate development by allowing fast turnaround and refactoring with confidence. There are bad unit tests, of course, that are highly coupled with the code that they test. I have written tens of thousands of unit tests in my career, and by far the most useful ones are the ones that point you to the exact subsystem in which a bug you've just introduced during refactoring/re-implementing lies. Bad unit tests are brittle "change detectors" and make you constantly adjust them for no real reason. I find mocks and interaction-based tests (rather than simple input/output tests) are this way. I flat-out do not use mocks for this very reason. They are intricate, involved, and take huge amounts of time to load context when they fail. Don't write such tests. If your unit tests are garbage, it's because they were not thought out very much and commit sins.

bcrosby95 5y ago
> I flat-out do not use mocks for this very reason. They are intricate, involved, and take huge amounts of time to load context when they fail. Don't write such tests. This is why I prefer integration tests. Most of my projects interact heavily with a database. Making it an integration test unless I mock it out. Which I don't do, because in my experience, doing so tends to hide lots of bugs.
- titzer 5y ago
  > This is why I prefer integration tests. IMO integration tests are great for making sure that your system actually functions with all its parts together. Nothing but an integration test can do that. But unit tests serve a different role: reducing the scope of the system under test so that you can pinpoint exactly what part failed when it does fail. They help you iterate quickly in refactoring, and when something goes wrong in production (or a bug report from the field), they narrow the cone of uncertainty by eliminating from consideration the parts of the system that are working correctly. In my experience, this rapidly cuts down on the amount of wild goose chases one goes on and overall makes a team more efficient. Then once a bug in the wild is found, the system responsible for the failure gets new regression tests and unit tests. Sometimes, I wrote unit tests for units I am suspicious of as I am debugging, and even if those tests don't find a bug, I might keep them. I worked for 6+ years on V8 and the parts of the system that were the least unit tested seemed to be home to far more production issues than others. It also helps a lot to keep your unit tests running super fast. Under 1 second is ideal.
deleted 5y ago
[deleted]
thom 5y ago
I think I agree with this, although I also think a lot of people would still call this integration testing. Basically a lot of code units are implementation details and you want to be free to refactor them fearlessly without having to write double the code each time tightly unit testing each part. Much better to narrow your system to a few well defined boundaries that deliver some clear value. The code behind that interface can be as deep as you like but you don’t really need to push tests down into the depths. As I say, some people will still think of this as integration testing just because there’s a lot of moving parts working together. But often you’re just testing fairly pure code with no external or IO dependencies so I’m happy thinking of these as unit tests still. Where those types of dependencies exist I do still mock though.

Rd6n6 5y ago

Except for unusually complex logic, almost every bug I’ve ever seen has been configuration or integration related. I do unit tests in safety critical software, but if I actually need to catch a bug I usually worry more about integration or end to end tests