4 ms·

> post-facto automated testing just locks in the codebase to behave exactly as it did before the tests What, in your opinion, is the purpose of automated tests

> post-facto automated testing just locks in the codebase to behave exactly as it did before the tests What, in your opinion, is the purpose of automated tests?

wesselbindt 3y ago
I can't answer for your parent, but for me it is the following. The main purpose of a test suite is to detect regressions in intended behavior. In a good test suite, if a test fails, then I can be sure that some part of the actual system would fail if I push this to prod. Since the set of intended behavior is only a small fraction of all actual behavior, if your test suite pins down all actual behavior, rather than just the intended behavior, you end up with a very brittle test suite. That is to say, a test failing doesn't tell you "this code is broken", it just tells you "this code changed".
evilduck 3y ago
This is nuanced, but I try for tests that ensure the business rules (or product features) of the code are being exercised without enforcing specific implementation or style. Sometimes that does mean being highly specific, like when you are conforming to standards or specifications or providing a contract like an API used by third parties, but other times you don’t care how the sausage is made you just need to ensure sausage is getting made. I don’t actually know how the article approached this but again in my own experience, when code is written without testing in mind from the beginning and you just go wild adding them later you instead get tests that enforce very specific coding styles, tests that enforce that the dependency graph is immutable, that internal data formats must never change, etc. If you can add tests without intimately understanding the business reason the code exists you’re more likely writing tests that just lock in implementation. This type of testing also means that every imaginable change will break dozens of tests and puts people in the mindset of just updating the test to pass instead of considering it a real problem because the tests stop giving meaningful information when they fail because literally everything you do will cause a slew of failures. That’s helpful in knowing the blast radius of your change but it multiplies the cost of change instead of reducing it.

> post-facto automated testing just locks in the codebase to behave exactly as it did before the tests What, in your opinion, is the purpose of automated tests?

wesselbindt 3y ago

I can't answer for your parent, but for me it is the following. The main purpose of a test suite is to detect regressions in intended behavior. In a good test suite, if a test fails, then I can be sure that some part of the actual system would fail if I push this to prod. Since the set of intended behavior is only a small fraction of all actual behavior, if your test suite pins down all actual behavior, rather than just the intended behavior, you end up with a very brittle test suite. That is to say, a test failing doesn't tell you "this code is broken", it just tells you "this code changed".

evilduck 3y ago

This is nuanced, but I try for tests that ensure the business rules (or product features) of the code are being exercised without enforcing specific implementation or style. Sometimes that does mean being highly specific, like when you are conforming to standards or specifications or providing a contract like an API used by third parties, but other times you don’t care how the sausage is made you just need to ensure sausage is getting made. I don’t actually know how the article approached this but again in my own experience, when code is written without testing in mind from the beginning and you just go wild adding them later you instead get tests that enforce very specific coding styles, tests that enforce that the dependency graph is immutable, that internal data formats must never change, etc. If you can add tests without intimately understanding the business reason the code exists you’re more likely writing tests that just lock in implementation. This type of testing also means that every imaginable change will break dozens of tests and puts people in the mindset of just updating the test to pass instead of considering it a real problem because the tests stop giving meaningful information when they fail because literally everything you do will cause a slew of failures. That’s helpful in knowing the blast radius of your change but it multiplies the cost of change instead of reducing it.