5 ms·

How to write a bug report that actually gets resolved – and why everyone should

45 points by groovykid 10y ago

cheiVia0 10y ago
https://anarc.at/blog/2016-10-14-bug-reporting/ https://anarc.at/blog/2016-10-14-bug-reporting/
zenincognito 10y ago
Reporting the bug is important but highlighting what are the implications of the bug is equally important. With one simple string , I was able to find 45 bugs in different software and every time I reported the bugs I decided to tell them what were the implications of the bugs. With no professional background in security, I was able to claim 18K in bug bounty rewards by being professional, clear and providing clear implications of the bug. It is quite possible that people fixing the bugs may not be well versed in some components of the bug and are likely to not understand the implication of the bugs until you tell them so.
- scoot 10y ago
  Good old SQL injection?
  - zenincognito 10y ago
    Good old >'>"><img src=x onerror=alert(1)>
arbesfeld 10y ago
Great post. I'm surprised that this topic doesn't get visited more given how much time we [developers] spend on bug fixing. Shameless self-promotion: we are working on LogRocket [1] specifically so that you don't have to ask users for steps to reproduce. [1] https://logrocket.com https://logrocket.com
dbg31415 10y ago
https://getmarker.io/ https://getmarker.io/ These guys are doing bug capture well.
- groovykid 10y ago
  That looks pretty fantastic. Definitely worth checking out.
delecti 10y ago
It's absolutely stunning how many testers I've worked with didn't understand the "Preconditions/Actions/Expected/Actual" as a bare minimum for a bug report. > Avoid using ambiguous words like “broken” or “not working.” Holy crap yes, this. So many infuriating waste of time bugs. And adding screenshots is good to call out. "X looks wrong", is useless, "X looks wrong <screenshot attached>" might still be useless, but at least you're trying.
- mikestew 10y ago
  It's absolutely stunning how many testers I've worked with didn't understand the "Preconditions/Actions/Expected/Actual" as a bare minimum for a bug report. What I've never understood is how testers with multiple years of experience continue to get away with this. To me it's like checking in code that doesn't compile: you can't even do the bare minimum required by your job. On the rare occasions I've had folks like that on my team, I've emphasized the importance of easy-to-replicate bug reports by pointing out that dev has to repro it in order to fix it, PM might have to repro in order to triage, another tester might have to repro in order to close the bug when you're on vacation, etc. Now, because of your shitty bug report, how much time has been wasted because your steps-to-repro are poor or non-existent? Me, I'm personally embarrassed when a bug comes back "no repro" because I half-assed the bug report. Let alone getting into things like, copy-and-paste the damned call stack that's staring you in the face. And if you're a professional tester to whom we pay good money, and your "observed" includes "broken" or "doesn't work" with no further description, your future with this company might be in jeopardy.
  - T-hawk 10y ago
    I had a previous job where the testers would always do that, just report "broken" or "doesn't work" for just about everything. I eventually figured out that they didn't actually want bugs fixed. They wanted to have lots of bug reports open and the application looking broken all over the place. Why? To justify the jobs and headcount and existence of the QA department. The QA managers would even resist movements towards automated testing, to protect their own turf and the employment of the far-too-many manual testers. I eventually got quite used to the disappointment in a tester's face when he had to close a bug report because it had been fixed. Classic case of "if you're being paid for a problem, the last thing you want to do is solve it."
bahjoite 10y ago
According to the flow chart, one doesn't need to "Look for an existing bug report before reporting the bug" if one doesn't like the software.
- bjterry 10y ago
  Seems like a pretty passive-aggressive way to punish the developers for a piece of software you don't like.
MakeUsersWant 10y ago
For example, an essential user action requires 3 cryptically named menu items, deeply hidden in 5 layers of menus, each of with takes ages to open. Does that count as a bug? Users will simply fail at their job. But for many (open-source) developers, only a segfault counts. I have stopped reporting such things.
antipasta 10y ago
After years of vague bug reports at various places we've worked, my 2 cofounders and I built a tool that records video of a bug, alongside network traffic and JavaScript logs. That way even with a vague description, and no steps to reproduce, we can get a much better idea of what a customer is trying to report to us. Would love for anyone interested to check it out: https://www.bugreplay.com https://www.bugreplay.com
- antipasta 10y ago
  Just dawned on me that a demo might be better than just linking to the marketing site :) Here is a BugReplay recording demonstrating an ebay security vulnerability that we recorded awhile back: https://app.bugreplay.com/shared/report/3efa632d-5b51-45f1-aa0f-8f5503d9c663 https://app.bugreplay.com/shared/report/3efa632d-5b51-45f1-a...
roadman 10y ago
I feel for you and the state of the QA industry. If you need someone efficient please contact me servicewareqa,gmail).

45 points by groovykid 10y ago

https://anarc.at/blog/2016-10-14-bug-reporting/ https://anarc.at/blog/2016-10-14-bug-reporting/

Reporting the bug is important but highlighting what are the implications of the bug is equally important. With one simple string , I was able to find 45 bugs in different software and every time I reported the bugs I decided to tell them what were the implications of the bugs. With no professional background in security, I was able to claim 18K in bug bounty rewards by being professional, clear and providing clear implications of the bug. It is quite possible that people fixing the bugs may not be well versed in some components of the bug and are likely to not understand the implication of the bugs until you tell them so.

scoot 10y ago
Good old SQL injection?
- zenincognito 10y ago
  Good old >'>"><img src=x onerror=alert(1)>

arbesfeld 10y ago

Great post. I'm surprised that this topic doesn't get visited more given how much time we [developers] spend on bug fixing. Shameless self-promotion: we are working on LogRocket [1] specifically so that you don't have to ask users for steps to reproduce. [1] https://logrocket.com https://logrocket.com

dbg31415 10y ago

https://getmarker.io/ https://getmarker.io/ These guys are doing bug capture well.

groovykid 10y ago
That looks pretty fantastic. Definitely worth checking out.

delecti 10y ago

It's absolutely stunning how many testers I've worked with didn't understand the "Preconditions/Actions/Expected/Actual" as a bare minimum for a bug report. > Avoid using ambiguous words like “broken” or “not working.” Holy crap yes, this. So many infuriating waste of time bugs. And adding screenshots is good to call out. "X looks wrong", is useless, "X looks wrong <screenshot attached>" might still be useless, but at least you're trying.

mikestew 10y ago
It's absolutely stunning how many testers I've worked with didn't understand the "Preconditions/Actions/Expected/Actual" as a bare minimum for a bug report. What I've never understood is how testers with multiple years of experience continue to get away with this. To me it's like checking in code that doesn't compile: you can't even do the bare minimum required by your job. On the rare occasions I've had folks like that on my team, I've emphasized the importance of easy-to-replicate bug reports by pointing out that dev has to repro it in order to fix it, PM might have to repro in order to triage, another tester might have to repro in order to close the bug when you're on vacation, etc. Now, because of your shitty bug report, how much time has been wasted because your steps-to-repro are poor or non-existent? Me, I'm personally embarrassed when a bug comes back "no repro" because I half-assed the bug report. Let alone getting into things like, copy-and-paste the damned call stack that's staring you in the face. And if you're a professional tester to whom we pay good money, and your "observed" includes "broken" or "doesn't work" with no further description, your future with this company might be in jeopardy.
- T-hawk 10y ago
  I had a previous job where the testers would always do that, just report "broken" or "doesn't work" for just about everything. I eventually figured out that they didn't actually want bugs fixed. They wanted to have lots of bug reports open and the application looking broken all over the place. Why? To justify the jobs and headcount and existence of the QA department. The QA managers would even resist movements towards automated testing, to protect their own turf and the employment of the far-too-many manual testers. I eventually got quite used to the disappointment in a tester's face when he had to close a bug report because it had been fixed. Classic case of "if you're being paid for a problem, the last thing you want to do is solve it."

bahjoite 10y ago

According to the flow chart, one doesn't need to "Look for an existing bug report before reporting the bug" if one doesn't like the software.

bjterry 10y ago
Seems like a pretty passive-aggressive way to punish the developers for a piece of software you don't like.

MakeUsersWant 10y ago

For example, an essential user action requires 3 cryptically named menu items, deeply hidden in 5 layers of menus, each of with takes ages to open. Does that count as a bug? Users will simply fail at their job. But for many (open-source) developers, only a segfault counts. I have stopped reporting such things.

antipasta 10y ago

After years of vague bug reports at various places we've worked, my 2 cofounders and I built a tool that records video of a bug, alongside network traffic and JavaScript logs. That way even with a vague description, and no steps to reproduce, we can get a much better idea of what a customer is trying to report to us. Would love for anyone interested to check it out: https://www.bugreplay.com https://www.bugreplay.com

antipasta 10y ago
Just dawned on me that a demo might be better than just linking to the marketing site :) Here is a BugReplay recording demonstrating an ebay security vulnerability that we recorded awhile back: https://app.bugreplay.com/shared/report/3efa632d-5b51-45f1-aa0f-8f5503d9c663 https://app.bugreplay.com/shared/report/3efa632d-5b51-45f1-a...

roadman 10y ago

I feel for you and the state of the QA industry. If you need someone efficient please contact me servicewareqa,gmail).