3 ms·

My argument against defect rate metrics is much simpler: the really scary situations are the ones where no one is looking closely enough to file bugs in the fir

My argument against defect rate metrics is much simpler: the really scary situations are the ones where no one is looking closely enough to file bugs in the first place. That’s where you find out it hasn’t worked at all in six months. Whereas the functionality subject to many discerning eyes filing bugs about every little thing is probably doing pretty well.

MarkSweep 7y ago
This reminds me of a good example from work: the emergency stop button on an industrial robot. By definition, the customer should not be pushing it. Pushing it means something has gone terribly wrong. But when they push it, they expect certain things to happen and they are unhappy when they don't. (Your typically industrial robot does not rely on software for safety. However software can make the robot respond more gracefully.)
- honestoHeminway 7y ago
  We once tested industrial robot fences- those fences do not stop the robot, if you deactivate the safety break zone- the robot goes clean through. All variants. They are there to prevent humans from entering. PS: Warstory seems familiar? Sebastian? Olli?
kabdib 7y ago
Been there. That time at Apple when we were sitting in a meeting, fat and happy about the bug count in a new C compiler and how ready it looked to ship, when it was revealed that a Q/A person had turned the floating point tests off six months earlier "because none of them were passing" and hadn't turned them back on. We all learned a lot about how floating point worked in the next couple of months.
shanemhansen 7y ago
I worked on a project where this came up. We had some product features that weren't really used. We continued to talk about all the benefits of those features and when the business had renewed interest in that area we jumped to turning those features on. Only to find out they barely worked. FWIW the unit tests were pretty solid but the integrations weren't and the graceful degradation story was a little too good. So we went from no complaints to having an entire department breathing down our next because all of the sudden people started giving a shit about our product (web performance improvements).

MarkSweep 7y ago

This reminds me of a good example from work: the emergency stop button on an industrial robot. By definition, the customer should not be pushing it. Pushing it means something has gone terribly wrong. But when they push it, they expect certain things to happen and they are unhappy when they don't. (Your typically industrial robot does not rely on software for safety. However software can make the robot respond more gracefully.)

honestoHeminway 7y ago
We once tested industrial robot fences- those fences do not stop the robot, if you deactivate the safety break zone- the robot goes clean through. All variants. They are there to prevent humans from entering. PS: Warstory seems familiar? Sebastian? Olli?

kabdib 7y ago

Been there. That time at Apple when we were sitting in a meeting, fat and happy about the bug count in a new C compiler and how ready it looked to ship, when it was revealed that a Q/A person had turned the floating point tests off six months earlier "because none of them were passing" and hadn't turned them back on. We all learned a lot about how floating point worked in the next couple of months.

shanemhansen 7y ago

I worked on a project where this came up. We had some product features that weren't really used. We continued to talk about all the benefits of those features and when the business had renewed interest in that area we jumped to turning those features on. Only to find out they barely worked. FWIW the unit tests were pretty solid but the integrations weren't and the graceful degradation story was a little too good. So we went from no complaints to having an entire department breathing down our next because all of the sudden people started giving a shit about our product (web performance improvements).