3 ms·

That's a great ideal. But often the code has to be substantially rearchitected to allow for automated testing like this (especially for those difficult bugs, a

That's a great ideal. But often the code has to be substantially rearchitected to allow for automated testing like this (especially for those difficult bugs, as you mention), and when the fix only requires changing a couple lines of code, it's not worth the risk. Although it would be great if we went through the exercise of "how would we write an automated test for this?" at a high level, even if it doesn't get implemented. That might get people thinking about how to architect their code so that it's more testable from the get-go.

beat 10y ago
If the code is written with a proper testing suite from the start, it's not hard. If you're dealing with a legacy code base that doesn't have a testing harness (I've done this a lot), then the first thing you should be doing before making changes is writing tests for them! Every time I do the "I don't need a test for this, it's just an obvious little bug" thing, and I implement a test-free fix, it fails! Every. Single. Time. Maybe it's because I'm stupid, but I like to think I'm above average. So if above-average programmers have a high risk of releasing "bug fixes" that don't actually fix correctly (or cause other problems) because they didn't write good tests, that really turns the "not worth the risk" response on its head. It's not that it's not worth the risk to write tests. It's not worth the risk to not write tests!
- tehlike 10y ago
  That's the part I meant when i said "known regressions". If there is a bug, you write the test, it doesn't happen again... Until next refactor, because most often you'll find that the tests were written at a lower level, instead of a functional test (that tests the e2e functionality) it was written to test individual pieces of SUT such as individual methods in a class. This happens quite a lot. Devs will prefer unit tests over more complex e2e tests which takes a lot more effort to write (because they are slow, flaky, more complex). Any friction in this process will make the dev prefer the easier way to test (which does not mean future proof).

beat 10y ago

If the code is written with a proper testing suite from the start, it's not hard. If you're dealing with a legacy code base that doesn't have a testing harness (I've done this a lot), then the first thing you should be doing before making changes is writing tests for them! Every time I do the "I don't need a test for this, it's just an obvious little bug" thing, and I implement a test-free fix, it fails! Every. Single. Time. Maybe it's because I'm stupid, but I like to think I'm above average. So if above-average programmers have a high risk of releasing "bug fixes" that don't actually fix correctly (or cause other problems) because they didn't write good tests, that really turns the "not worth the risk" response on its head. It's not that it's not worth the risk to write tests. It's not worth the risk to not write tests!

tehlike 10y ago
That's the part I meant when i said "known regressions". If there is a bug, you write the test, it doesn't happen again... Until next refactor, because most often you'll find that the tests were written at a lower level, instead of a functional test (that tests the e2e functionality) it was written to test individual pieces of SUT such as individual methods in a class. This happens quite a lot. Devs will prefer unit tests over more complex e2e tests which takes a lot more effort to write (because they are slow, flaky, more complex). Any friction in this process will make the dev prefer the easier way to test (which does not mean future proof).