3 ms·

Isn't that a bit of a red herring? Either I have automated tests that will take time to update every once in a while, or I have manual regression tests that I

Isn't that a bit of a red herring? Either I have automated tests that will take time to update every once in a while, or I have manual regression tests that I have to run for every code change. The automated test path seems to be the much less time intensive path, depending on how often you make big changes. On the other hand I really believe 100% unit test coverage is a bit of a waste of time. Unit tests should be reserved for algorithmic\calculative code, more mundane code logging\carting data from a to b type code should be covered by automated black box testing.

klingon78 6y ago
I meant upgrading application frameworks, OS, etc., so it’s not a red herring. Those can inflict a lot of changes if you have a lot of tests affected.
- stonemetal12 6y ago
  Thus automated test time + automated test maintenance is usually less than manual test time + manual test maintenance. Mostly because execution time for automated tests is so much shorter than manual execution time that the higher cost of automated test maintenance is recouped and then some. The only way for framework\OS changes to impact that is if they happen often enough to make automated test maintenance "weigh" more than all the time saved by no longer having to manually run your test suite.
  - klingon78 6y ago
    I don’t think we’re speaking the same language. I’m talking about how much time it takes to change the code in every unit test, etc. because you upgraded something that requires changing the test framework, and this library and that one, so you end up not only having to fix all of your production code, but all of the tests with their mocks, stubs, injections, annotations, xml config, etc. If you have thousands of tests, this can keep people from upgrading.
bluGill 6y ago
That depends on the tests. I've seen tests that are the code is implemented like X. I've seen tests that are the code uses some API to do X. I've seen tests that are the program does X via some user interface. What you want are tests that the program does X, regardless of what computer it runs on, or what the interfaces are. However this is impossible long term. You MUST assume something that is subject to change in the future. Some changes are more subject to change than others. Automated tests are an assertion that "this will never again change". You can make an educated guess as to what will and will not change, if you guess right then you don't need to rework your tests. When (not if!) you guess wrong you need to rework tests.

klingon78 6y ago

I meant upgrading application frameworks, OS, etc., so it’s not a red herring. Those can inflict a lot of changes if you have a lot of tests affected.

stonemetal12 6y ago
Thus automated test time + automated test maintenance is usually less than manual test time + manual test maintenance. Mostly because execution time for automated tests is so much shorter than manual execution time that the higher cost of automated test maintenance is recouped and then some. The only way for framework\OS changes to impact that is if they happen often enough to make automated test maintenance "weigh" more than all the time saved by no longer having to manually run your test suite.
- klingon78 6y ago
  I don’t think we’re speaking the same language. I’m talking about how much time it takes to change the code in every unit test, etc. because you upgraded something that requires changing the test framework, and this library and that one, so you end up not only having to fix all of your production code, but all of the tests with their mocks, stubs, injections, annotations, xml config, etc. If you have thousands of tests, this can keep people from upgrading.

bluGill 6y ago

That depends on the tests. I've seen tests that are the code is implemented like X. I've seen tests that are the code uses some API to do X. I've seen tests that are the program does X via some user interface. What you want are tests that the program does X, regardless of what computer it runs on, or what the interfaces are. However this is impossible long term. You MUST assume something that is subject to change in the future. Some changes are more subject to change than others. Automated tests are an assertion that "this will never again change". You can make an educated guess as to what will and will not change, if you guess right then you don't need to rework your tests. When (not if!) you guess wrong you need to rework tests.