4 ms·

> "Is the test simpler than the implementation?" Related, but writing a regression test for every little bug you fix isn't always worth the time either, like w

> "Is the test simpler than the implementation?" Related, but writing a regression test for every little bug you fix isn't always worth the time either, like when the test is complex to write, the bug isn't likely to happen again, and the consequences of the bug aren't costly. It's rare to write a test to confirm a typo in some copy/text was fixed for example. Sounds obvious, but I've been on teams where you're seen as a cowboy if you even try to discuss the downsides/costs of testing.

vladms 3y ago
I think you make the assumption that the person writing the test will be able to write a test that will trigger only when the same bug would happen. This could happen for some very narrow unit tests (as in the examples), but in real life there will be many tests involving multiple functions (to reproduce the same situation/code path) and then the tests can prevent other bugs. (put it another way: if the programmer introduced a bug by mistake, probably by writing a test it will test more than wanted by mistake...)
- seanwilson 3y ago
  > I think you make the assumption that the person writing the test will be able to write a test that will trigger only when the same bug would happen. I'm aware of this, it still comes down to weighing up the costs vs benefits. It's not practical to write a test for every small behaviour of a complex app.
codingdave 3y ago
I agree that testing for a typo is silly, but the consequences of a repeated bug aren't technical, they are about customer service. If your customers see the same bug repeatedly occur on multiple releases, they lose trust in your product. That is a huge consequence. And one easily remedied by a test for that bug - sure, it offers little technical value, but is a solid guardrail against, "Didn't you guys just fix this??", for the same surface-level symptom that could be caused by multiple technical root causes. After all, if a customer sees the same widget break the same way 3 releases in a row, the last thing they want to hear is: "Well, actually, that just looks like the same bug, but really..."
- seanwilson 3y ago
  > If your customers see the same bug repeatedly occur on multiple releases, they lose trust in your product. That is a huge consequence. I mentioned to weigh up how likely the bug will happen again and the consequences of that specific bug. A typo isn't likely to impact trust nearly as much as a crash. In what scenarios do you not write tests? Have you been in a situation where writing the test would be a lot more work than it's worth?

vladms 3y ago

I think you make the assumption that the person writing the test will be able to write a test that will trigger only when the same bug would happen. This could happen for some very narrow unit tests (as in the examples), but in real life there will be many tests involving multiple functions (to reproduce the same situation/code path) and then the tests can prevent other bugs. (put it another way: if the programmer introduced a bug by mistake, probably by writing a test it will test more than wanted by mistake...)

seanwilson 3y ago
> I think you make the assumption that the person writing the test will be able to write a test that will trigger only when the same bug would happen. I'm aware of this, it still comes down to weighing up the costs vs benefits. It's not practical to write a test for every small behaviour of a complex app.

codingdave 3y ago

I agree that testing for a typo is silly, but the consequences of a repeated bug aren't technical, they are about customer service. If your customers see the same bug repeatedly occur on multiple releases, they lose trust in your product. That is a huge consequence. And one easily remedied by a test for that bug - sure, it offers little technical value, but is a solid guardrail against, "Didn't you guys just fix this??", for the same surface-level symptom that could be caused by multiple technical root causes. After all, if a customer sees the same widget break the same way 3 releases in a row, the last thing they want to hear is: "Well, actually, that just looks like the same bug, but really..."

seanwilson 3y ago
> If your customers see the same bug repeatedly occur on multiple releases, they lose trust in your product. That is a huge consequence. I mentioned to weigh up how likely the bug will happen again and the consequences of that specific bug. A typo isn't likely to impact trust nearly as much as a crash. In what scenarios do you not write tests? Have you been in a situation where writing the test would be a lot more work than it's worth?