4 ms·

I disagree: https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing-is-waste/ https://henrikwarne.com/2014/09/04/a-response-to-why-most-un...

Illniyar 9y ago
This. A thousand times this. The debate about testing should stop being about what type of testing os the best and start being about what type of test to use where.
crdoconnor 9y ago
"In my experience, unit tests are most valuable when you use them for algorithmic logic. They are not particularly useful for code that is more coordinating in its nature." I agree in principle, but IME: * This kind of algorithmic code often makes up a fairly small proportion (~5%) of code written for most business driven applications - it's usually mostly coordination. Some domains may be different (and for them, unit testing will appear to be much more effective), but I think this is the norm. * Integration tests can test algorithmic acceptably well, but unit tests do not test integration code in an acceptable fashion. * Where you have algorithmic and integration code smushed together (this type of technical debt is, sadly, the norm 'in the wild'), again, integration tests work acceptably well whereas unit tests require a mess of mocks. * Integration tests do not have to be high level or slow. "Refactoring breaks tests. Sometimes when you refactor code, you break tests. But my experience is that this is not a big problem. For example, a method signature changes, so you have to go through and add an extra parameter in all tests where it is called. This can often be done very quickly, and it doesn’t happen very often. This sounds like a big problem in theory, but in practice it isn’t." This is a big problem in practice when you are working on large scale code bases in which the developers have not been super strict about decoupling everything.
- RHSeeger 9y ago
  > Integration tests can test algorithmic acceptably well, but unit tests do not test integration code in an acceptable fashion. True, but you can have both types of tests, which lets you test algorithms (unit tests) and higher level combinations (integration tests) both in the best way. The right tool for the right job. > Where you have algorithmic and integration code smushed together (this type of technical debt is, sadly, the norm 'in the wild'), again, integration tests work acceptably well whereas unit tests require a mess of mocks. In many cases, you'd be better off separating the logic, which would let you have both types of tests, each for a different part of the logic. > Integration tests do not have to be high level or slow. No, but they are a different tool.
  - crdoconnor 9y ago
    >True, but you can have both types of tests You can, but if your code is 95% coordination and 5% algorithmic, at most you'd want 5% unit tests. >In many cases, you'd be better off separating the logic Which is refactoring and if you're refactoring you need to have your code surrounded with tests to do it safely. And, once you've done that, if you've already written integration tests for that logic and they perform acceptably well, there may be no point in rewriting the integration test as a unit test.
- jononor 9y ago
  For well-defined algorithmic logic, I'd say one is better off mostly using existing code (usually via a library). If a library does not exist, write the function as stand-alone with tests exercising its API. Preferably release it as open source.
mannykannot 9y ago
There are a number of points here that I do not find convincing - for example: The combinatorial complexity argument does apply more (much more, in many cases) to unit testing than to testing at higher level of abstractions, broadly for the same reasons that abstraction ameliorates the design-complexity problem. Consider a program using heuristic methods to find a near-optimal solution to an NP problem: not only would testing that it produces valid solutions at the program level be a lot simpler than unit-testing all its components, no amount of the latter would establish that any of its solutions are valid. The point is not that integration testing allows you to cover the whole state-space, but that it is a more effective way to select what you test. The point of assertions is that they are tested in the execution of integration tests. It has been my experience that most of the effort expended in the debug-fix cycle is a result of errors in decomposing the problem (or, equivalently, composing the units) - for example, an incorrect assumption about the data a unit uses, rather than the handling of the data in accordance with the assumptions made about it. Discrepancies between what is expected of a component and what it does also seem to be common, such as when one component is predicated on a constraint being respected by another, but the latter does not always do so.

Illniyar 9y ago

This. A thousand times this. The debate about testing should stop being about what type of testing os the best and start being about what type of test to use where.

crdoconnor 9y ago

"In my experience, unit tests are most valuable when you use them for algorithmic logic. They are not particularly useful for code that is more coordinating in its nature." I agree in principle, but IME: * This kind of algorithmic code often makes up a fairly small proportion (~5%) of code written for most business driven applications - it's usually mostly coordination. Some domains may be different (and for them, unit testing will appear to be much more effective), but I think this is the norm. * Integration tests can test algorithmic acceptably well, but unit tests do not test integration code in an acceptable fashion. * Where you have algorithmic and integration code smushed together (this type of technical debt is, sadly, the norm 'in the wild'), again, integration tests work acceptably well whereas unit tests require a mess of mocks. * Integration tests do not have to be high level or slow. "Refactoring breaks tests. Sometimes when you refactor code, you break tests. But my experience is that this is not a big problem. For example, a method signature changes, so you have to go through and add an extra parameter in all tests where it is called. This can often be done very quickly, and it doesn’t happen very often. This sounds like a big problem in theory, but in practice it isn’t." This is a big problem in practice when you are working on large scale code bases in which the developers have not been super strict about decoupling everything.

RHSeeger 9y ago
> Integration tests can test algorithmic acceptably well, but unit tests do not test integration code in an acceptable fashion. True, but you can have both types of tests, which lets you test algorithms (unit tests) and higher level combinations (integration tests) both in the best way. The right tool for the right job. > Where you have algorithmic and integration code smushed together (this type of technical debt is, sadly, the norm 'in the wild'), again, integration tests work acceptably well whereas unit tests require a mess of mocks. In many cases, you'd be better off separating the logic, which would let you have both types of tests, each for a different part of the logic. > Integration tests do not have to be high level or slow. No, but they are a different tool.
- crdoconnor 9y ago
  >True, but you can have both types of tests You can, but if your code is 95% coordination and 5% algorithmic, at most you'd want 5% unit tests. >In many cases, you'd be better off separating the logic Which is refactoring and if you're refactoring you need to have your code surrounded with tests to do it safely. And, once you've done that, if you've already written integration tests for that logic and they perform acceptably well, there may be no point in rewriting the integration test as a unit test.
jononor 9y ago
For well-defined algorithmic logic, I'd say one is better off mostly using existing code (usually via a library). If a library does not exist, write the function as stand-alone with tests exercising its API. Preferably release it as open source.

mannykannot 9y ago

There are a number of points here that I do not find convincing - for example: The combinatorial complexity argument does apply more (much more, in many cases) to unit testing than to testing at higher level of abstractions, broadly for the same reasons that abstraction ameliorates the design-complexity problem. Consider a program using heuristic methods to find a near-optimal solution to an NP problem: not only would testing that it produces valid solutions at the program level be a lot simpler than unit-testing all its components, no amount of the latter would establish that any of its solutions are valid. The point is not that integration testing allows you to cover the whole state-space, but that it is a more effective way to select what you test. The point of assertions is that they are tested in the execution of integration tests. It has been my experience that most of the effort expended in the debug-fix cycle is a result of errors in decomposing the problem (or, equivalently, composing the units) - for example, an incorrect assumption about the data a unit uses, rather than the handling of the data in accordance with the assumptions made about it. Discrepancies between what is expected of a component and what it does also seem to be common, such as when one component is predicated on a constraint being respected by another, but the latter does not always do so.