The sad fact of E2E is that the tests genuinely find broken stuff. The “false negative” test results usually just mean false as in “something was broken, just

The sad fact of E2E is that the tests genuinely find broken stuff. The “false negative” test results usually just mean false as in “something was broken, just not what the CI claimed was broken.” It’s could be anything, so you need automatic specificity as to what’s broken (hard) or buy-in from the entire organisation to be on standby for finding broken stuff (also hard.) “Anything” as in if your external DNS provider has 1 of 10 resolvers with an out of date zonefile, or a dodgy switch port to that particular resolver. It’s hard but if it’s broken then it’s likely it is a real issue one of your end users is also experiencing. A commitment to E2E is committing to a level of quality across your entire infrastructure that few people are prepared to own.