4 ms·

I can't imagine a single developer would have the power to make this change either? I guess it's possible that filtering has existed for a long time for very s

by eterm 9y ago

I can't imagine a single developer would have the power to make this change either? I guess it's possible that filtering has existed for a long time for very specific queries and 'gun' was added to a list. It seems weird that this feature could be put live in this state getting passed QA though. That makes me worried what other changes could get live without much oversight.

gfodor 9y ago
For an ecommerce site such as Google Shopping it goes beyond QA -- incremental rollouts should have triggered alerts noting this as causing a drop in recall (to effectively zero) for a large category of high traffic queries that convert (presumably there are some, like "nerf gun" or "glue gun", given that "gun" is an extremely common term that has high purchase intent for non-weapons) and it should have been prevented from being deployed by automation. Presumably if they are doing things correctly there are probably several overrides that needed to be enabled to let this change out. Or, they're not doing things correctly. In nearly any other situation, automated systems would recognize "change goes out -> large number of queries suddenly return zero results" as a major catastrophic bug and prevent it from being deployed.
- tinalumfoil 9y ago
  I'm hesitant to accept this was intentional because QA process, since this change was clearly made by someone who didn't know what they were doing. As another user pointed out, the word burgundy triggers the filter. And if you spell something incorrectly the "did you mean" results are unaffected. I'm imagining a higher level manager with very little technical expertise (just enough to write an if statement) pushing this into prod against Google's wishes, or maybe by accident.
  - gfodor 9y ago
    My point was that at Google, a single person should not be able to break things to this degree. The automation should roll it back automatically. This is, technically due to the drop in recall, a bug, unless it is overridden as a special case so it is not recognized as a bug. To break things this bad you not only should have to make the change to the code but you should also have to "sign off" on the dynamics flagged by the X% rollout test as being acceptable somehow. So no matter what this paints Google extremely poorly -- either Google Shopping is one git commit away from going down completely, or they're placing shipping a hacky, poorly executed political statement higher on their priorities than potentially damaging the livelihood of those who are collateral damage.

by eterm 9y ago

gfodor 9y ago

For an ecommerce site such as Google Shopping it goes beyond QA -- incremental rollouts should have triggered alerts noting this as causing a drop in recall (to effectively zero) for a large category of high traffic queries that convert (presumably there are some, like "nerf gun" or "glue gun", given that "gun" is an extremely common term that has high purchase intent for non-weapons) and it should have been prevented from being deployed by automation. Presumably if they are doing things correctly there are probably several overrides that needed to be enabled to let this change out. Or, they're not doing things correctly. In nearly any other situation, automated systems would recognize "change goes out -> large number of queries suddenly return zero results" as a major catastrophic bug and prevent it from being deployed.

tinalumfoil 9y ago
I'm hesitant to accept this was intentional because QA process, since this change was clearly made by someone who didn't know what they were doing. As another user pointed out, the word burgundy triggers the filter. And if you spell something incorrectly the "did you mean" results are unaffected. I'm imagining a higher level manager with very little technical expertise (just enough to write an if statement) pushing this into prod against Google's wishes, or maybe by accident.
- gfodor 9y ago
  My point was that at Google, a single person should not be able to break things to this degree. The automation should roll it back automatically. This is, technically due to the drop in recall, a bug, unless it is overridden as a special case so it is not recognized as a bug. To break things this bad you not only should have to make the change to the code but you should also have to "sign off" on the dynamics flagged by the X% rollout test as being acceptable somehow. So no matter what this paints Google extremely poorly -- either Google Shopping is one git commit away from going down completely, or they're placing shipping a hacky, poorly executed political statement higher on their priorities than potentially damaging the livelihood of those who are collateral damage.