9 ms·

Sometimes you’re in a hurry to fix a bug, so you write the functionality and ship it out fast after some manual testing, without an automated test reproducing t

Sometimes you’re in a hurry to fix a bug, so you write the functionality and ship it out fast after some manual testing, without an automated test reproducing the bug. That’s okay! Write the test afterwards in some branch, and after committing the tests, make an extra commit that undoes the bug fix. Let your CI run it and confirm it fails there, then bring back the bug fix on the next commit. That way, you can have flexibility to fix things fast, but still keep the regression test that’s proven to check for it around for the future.

rad_gruchalski 6y ago
> That’s okay! The effort put into manual tests can be committed directly to adding a test case. Furthermore, a fix should be accepted only when a full test suite with the new tests pass. Such manual testing, depending on what the issue was, may result in a regression in other parts of software. Just a thought...
afarrell 6y ago
> to fix things fast Note that if you see _another_ developer taking the time to write a test when you wouldn't, it doesn't necessarily mean they are wasting time. As I am debugging something, I need to write myself a very clear description of my hypothesis of the steps to reproduce the bug -- otherwise it is hard to see incremental progress. I work faster if I can have the machine execute those steps.
- karmelapple 6y ago
  Fully agreed. I didn't intend to imply a developer writing a test is wasting time, either! Our team has been in situations where we're highly confident of the root cause, but we know creating a test to duplicate this might take hours, if not days. It might even be a fairly finicky scenario to try and setup. Rather than letting customers have to handle the negative consequence of our bug for hours, we'll make the change, run it through our existing test suite (to make sure we're not making yet more troubles for ourselves!), and then release it after another teammate reviews the change. But a test certainly will help with the confidence that the right thing was changed. I would definitely encourage writing a test if it's easy for your group to handle whatever negative consequences the bug existing in the wild is producing for as long as it takes to write a test.
solarengineer 6y ago
"That's okay!" Why is that Ok? How does the dev know that they've not undone anything else? Also, how does the dev know that the fix is complete? Or that it caters to the defect? This anxiety of pushing out at fix - at the risk of undoing other working functionality - ought to be addressed first. It is better (and safer) to get into the habit of writing a test to reproduce the defect and then write the fix. After all, if the fix appears trivial, then the test for the defect ought not to take too much time either. I have been able to get to this mindset with practice.
- KptMarchewa 6y ago
  >How does the dev know that they've not undone anything else? No one says that rest of the test suite is not run.
- kyralis 6y ago
  Because being dogmatic is exactly what causes people to start ignoring this sort of methodology. Pragmatism really does have to win out sometimes. Maybe you're in a hurry because the bug is causing active downtime. Some bugs really are "obvious" once they've failed and you're looking at the code. Maybe development of a proper test involves some test infrastructure work that's a larger undertaking than you have the opportunity for at the moment. Maybe you have a solid manual/QA testing system behind you, allowing you at least temporary assurance that your fix is valid. No team does everything perfectly all the time, and that's fine. The real question is what gets done about it afterward: is the technical debt that you've incurred paid down in a reasonable time frame?
  - karmelapple 6y ago
    This nicely encapsulates what I was hoping to say, but didn't take the time to write out. Thank you! > Maybe development of a proper test involves some test infrastructure work that's a larger undertaking than you have the opportunity for at the moment. This is something I've encountered many times. > Maybe you have a solid manual/QA testing system behind you, allowing you at least temporary assurance that your fix is valid. I would hope most people are doing this anyway, especially when a big production bug has been found.
tonyedgecombe 6y ago
Sometimes you’re in a hurry to fix a bug Most of the time I find it faster to write the test first. If you haven't got a test what are you going to do? Manually test?
- hnick 6y ago
  We tend to have a web focus on this site but not all IT is like that. If I have a data processing job that takes 3 hours and it fell over at the 1 hour mark (and let's say whoever wrote it didn't have the foresight to make it resume neatly, because that's added complexity that never got budgeted), I'm going to fix the obvious bug and kick off reprocessing immediately. Possibly after some messing around in a REPL to confirm how the code acts. While it's going, I can then do some manual tests and sanity checks and cancel/restart if necessary - but if not, I've gained a lot of time.
- deleted 6y ago
  [deleted]
- MockObject 6y ago
  And since everything usually takes longer than expected, the benefit of automating that manual testing is usually greater than expected.
dhosek 6y ago
Nope. With experience comes wisdom. I've had the situation happen when I was sure I knew what the bug was. I wrote a test that I expected to fail—and it passed. My diagnosis was incorrect and the bug was somewhere else. Had I not done this, I might have not only not fixed the bug, I would have likely introduced new bugs.

rad_gruchalski 6y ago

> That’s okay! The effort put into manual tests can be committed directly to adding a test case. Furthermore, a fix should be accepted only when a full test suite with the new tests pass. Such manual testing, depending on what the issue was, may result in a regression in other parts of software. Just a thought...

afarrell 6y ago

> to fix things fast Note that if you see _another_ developer taking the time to write a test when you wouldn't, it doesn't necessarily mean they are wasting time. As I am debugging something, I need to write myself a very clear description of my hypothesis of the steps to reproduce the bug -- otherwise it is hard to see incremental progress. I work faster if I can have the machine execute those steps.

karmelapple 6y ago
Fully agreed. I didn't intend to imply a developer writing a test is wasting time, either! Our team has been in situations where we're highly confident of the root cause, but we know creating a test to duplicate this might take hours, if not days. It might even be a fairly finicky scenario to try and setup. Rather than letting customers have to handle the negative consequence of our bug for hours, we'll make the change, run it through our existing test suite (to make sure we're not making yet more troubles for ourselves!), and then release it after another teammate reviews the change. But a test certainly will help with the confidence that the right thing was changed. I would definitely encourage writing a test if it's easy for your group to handle whatever negative consequences the bug existing in the wild is producing for as long as it takes to write a test.

solarengineer 6y ago

"That's okay!" Why is that Ok? How does the dev know that they've not undone anything else? Also, how does the dev know that the fix is complete? Or that it caters to the defect? This anxiety of pushing out at fix - at the risk of undoing other working functionality - ought to be addressed first. It is better (and safer) to get into the habit of writing a test to reproduce the defect and then write the fix. After all, if the fix appears trivial, then the test for the defect ought not to take too much time either. I have been able to get to this mindset with practice.

KptMarchewa 6y ago
>How does the dev know that they've not undone anything else? No one says that rest of the test suite is not run.
kyralis 6y ago
Because being dogmatic is exactly what causes people to start ignoring this sort of methodology. Pragmatism really does have to win out sometimes. Maybe you're in a hurry because the bug is causing active downtime. Some bugs really are "obvious" once they've failed and you're looking at the code. Maybe development of a proper test involves some test infrastructure work that's a larger undertaking than you have the opportunity for at the moment. Maybe you have a solid manual/QA testing system behind you, allowing you at least temporary assurance that your fix is valid. No team does everything perfectly all the time, and that's fine. The real question is what gets done about it afterward: is the technical debt that you've incurred paid down in a reasonable time frame?
- karmelapple 6y ago
  This nicely encapsulates what I was hoping to say, but didn't take the time to write out. Thank you! > Maybe development of a proper test involves some test infrastructure work that's a larger undertaking than you have the opportunity for at the moment. This is something I've encountered many times. > Maybe you have a solid manual/QA testing system behind you, allowing you at least temporary assurance that your fix is valid. I would hope most people are doing this anyway, especially when a big production bug has been found.

tonyedgecombe 6y ago

Sometimes you’re in a hurry to fix a bug Most of the time I find it faster to write the test first. If you haven't got a test what are you going to do? Manually test?

hnick 6y ago
We tend to have a web focus on this site but not all IT is like that. If I have a data processing job that takes 3 hours and it fell over at the 1 hour mark (and let's say whoever wrote it didn't have the foresight to make it resume neatly, because that's added complexity that never got budgeted), I'm going to fix the obvious bug and kick off reprocessing immediately. Possibly after some messing around in a REPL to confirm how the code acts. While it's going, I can then do some manual tests and sanity checks and cancel/restart if necessary - but if not, I've gained a lot of time.
deleted 6y ago
[deleted]
MockObject 6y ago
And since everything usually takes longer than expected, the benefit of automating that manual testing is usually greater than expected.

dhosek 6y ago

Nope. With experience comes wisdom. I've had the situation happen when I was sure I knew what the bug was. I wrote a test that I expected to fail—and it passed. My diagnosis was incorrect and the bug was somewhere else. Had I not done this, I might have not only not fixed the bug, I would have likely introduced new bugs.