3 ms·

Personally I like record/replay style testing where you save real network(or even hardware communication) info then commit it and replay it to get quick unit te

by rat87 2y ago

Personally I like record/replay style testing where you save real network(or even hardware communication) info then commit it and replay it to get quick unit tests. That plus local file based SQL tests

meowtimemania 2y ago
How do you handle when fields are added or removed? Do you have to rerecord all the affected tests?
- vkou 2y ago
  Yes. When you change behavior you should expect to re-record.
3np 2y ago
That's integration testing. Also valuable but complementary to actual unit tests.
- globular-toast 2y ago
  The "unit" in unit test refers to the test, not the code under test. High level tests that are isolated from other tests using mechanisms like fixtures, setup/teardown etc are still unit tests.
globular-toast 2y ago
Sure but the problem is they are slow and the combinatorial explosion of possibilities at the edge of your application will kill you. You can't enumerate every possible code path in an application of reasonable size. This is why a balanced approach is usually employed: a small number of representative happy/unhappy path integration tests to test the high level user feedback etc, with lower-level tests on the underlying layers and libraries etc. If you could just do it with high level tests, we'd all do it that way.
cratermoon 2y ago
In 1997 Cem Kaner wrote a paper titled "Improving the Maintainability of Automated Test Suites"[1]. One of the key pain points Kaner identified was the fragility of capture/replay tests. There's a reason why the test automation tools popular in the 90s fell out of favor: any time the system under test changes, the capture/replay tests will be out of sync. In addition, capture/replay tests aren't very good at finding defects. 1 https://kaner.com/pdfs/autosqa.pdf https://kaner.com/pdfs/autosqa.pdf

3 ms·

Personally I like record/replay style testing where you save real network(or even hardware communication) info then commit it and replay it to get quick unit te

by rat87 2y ago

meowtimemania 2y ago
How do you handle when fields are added or removed? Do you have to rerecord all the affected tests?
- vkou 2y ago
  Yes. When you change behavior you should expect to re-record.
3np 2y ago
That's integration testing. Also valuable but complementary to actual unit tests.
- globular-toast 2y ago
  The "unit" in unit test refers to the test, not the code under test. High level tests that are isolated from other tests using mechanisms like fixtures, setup/teardown etc are still unit tests.
globular-toast 2y ago
Sure but the problem is they are slow and the combinatorial explosion of possibilities at the edge of your application will kill you. You can't enumerate every possible code path in an application of reasonable size. This is why a balanced approach is usually employed: a small number of representative happy/unhappy path integration tests to test the high level user feedback etc, with lower-level tests on the underlying layers and libraries etc. If you could just do it with high level tests, we'd all do it that way.
cratermoon 2y ago
In 1997 Cem Kaner wrote a paper titled "Improving the Maintainability of Automated Test Suites"[1]. One of the key pain points Kaner identified was the fragility of capture/replay tests. There's a reason why the test automation tools popular in the 90s fell out of favor: any time the system under test changes, the capture/replay tests will be out of sync. In addition, capture/replay tests aren't very good at finding defects. 1 https://kaner.com/pdfs/autosqa.pdf https://kaner.com/pdfs/autosqa.pdf