3 ms·

This is the core point. A canary deployment that was not preceded by deploying data that activates the region of the binary in question will prove nothing usefu

This is the core point. A canary deployment that was not preceded by deploying data that activates the region of the binary in question will prove nothing useful at all, while promoting a false sense of security.

ethbr1 1y ago
The root problem is that a dev team didn't appropriately communicate criteria for testing their new feature. Which definitely seems like shortcut 'on to the next thing and ignore QA due diligence.'
- sagarm 1y ago
  Google does not separate dev and QA in general (I'm sure there are exceptions).
  - ethbr1 1y ago
    That sounds like a conflict of interest in practice. Who's ass is responsible when a dev team starts missing coverage in their tests, like happened here?
    - sagarm 1y ago
      ...the engineering team? I'm confused by this idea that engineering teams are not responsible for software quality. Who else would be? My experience with separate QA teams was not encouraging. They dramatically slowed down releases while failing to catch bugs. Their capability to automate was poor, so they were doing manual testing that provided little coverage especially compared to our comprehensive submit-blocking unit and integration tests.