5 ms·

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely” The point is that when you do integration tes

by pknopf 7y ago

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely” The point is that when you do integration tests, you will test the underlying classes the way in which it actually matters. You can't test every possible condition a unit can have, but you can test for the most likely.

crdoconnor 7y ago
This is my experience too. Realism in testing is criminally underrated while code coverage is criminally overrated. IME unit tests can only effectively substitute for integration tests where you're testing logical/algorithmic code with simple function inputs/outputs.
- 0xB31B1B 7y ago
  I somewhat agree, somewhat disagree. IMO the largest value I get from unit tests is the confidence that I can make changes and understand what breaks. Refactoring w/o unit tests makes me feel like I am flying blind
  - closeparen 7y ago
    The units we test are almost never big enough for internal refactoring. It’s the decomposition into units that wants refactoring, and there the test suite actively fights back (mock expectations in particular). If there were enough code involved in a test that we could meaningfully refactor it while keeping the test green, we would call it an integration test.
sulam 7y ago
This is no more or less likely in an integration test vs a unit test. One of the first things you learn in writing unit tests is that you should be selective in picking your inputs. This is even taught in those “trade schools” he subtly derides. You regularly see both public test cases and hidden test cases run against problem sets, at least that’s what my interns tell me.
- walshemj 7y ago
  Interesting so they don't teach testing by deliberately trying to break things, pasting Arabic text into an address field for example. When testing NAPI like to use edge cases like the company with the longest name eg "Donaudampfschiffahrtsgesellschaft" or the Famous Welsh location "Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" I am pretty sure I crashed an A17 clearpath mainframe by doing aggressive testing
  - waxjar 7y ago
    This is the sort of input that you'd selectively pick in a unit test, too. What you don't do is also picking inputs like "llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" too (all lowercase) :)
  - floriol 7y ago
    That's QA, not the sort of testing we are talking about. In case QA happens to find some bug like that, after the fix, we should include another test case with the formerly breaking one

by pknopf 7y ago

crdoconnor 7y ago

This is my experience too. Realism in testing is criminally underrated while code coverage is criminally overrated. IME unit tests can only effectively substitute for integration tests where you're testing logical/algorithmic code with simple function inputs/outputs.

0xB31B1B 7y ago
I somewhat agree, somewhat disagree. IMO the largest value I get from unit tests is the confidence that I can make changes and understand what breaks. Refactoring w/o unit tests makes me feel like I am flying blind
- closeparen 7y ago
  The units we test are almost never big enough for internal refactoring. It’s the decomposition into units that wants refactoring, and there the test suite actively fights back (mock expectations in particular). If there were enough code involved in a test that we could meaningfully refactor it while keeping the test green, we would call it an integration test.

sulam 7y ago

This is no more or less likely in an integration test vs a unit test. One of the first things you learn in writing unit tests is that you should be selective in picking your inputs. This is even taught in those “trade schools” he subtly derides. You regularly see both public test cases and hidden test cases run against problem sets, at least that’s what my interns tell me.

walshemj 7y ago
Interesting so they don't teach testing by deliberately trying to break things, pasting Arabic text into an address field for example. When testing NAPI like to use edge cases like the company with the longest name eg "Donaudampfschiffahrtsgesellschaft" or the Famous Welsh location "Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" I am pretty sure I crashed an A17 clearpath mainframe by doing aggressive testing
- waxjar 7y ago
  This is the sort of input that you'd selectively pick in a unit test, too. What you don't do is also picking inputs like "llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch" too (all lowercase) :)
- floriol 7y ago
  That's QA, not the sort of testing we are talking about. In case QA happens to find some bug like that, after the fix, we should include another test case with the formerly breaking one