3 ms·
Testing doesn't detect failure, it only detects the failure of a test. Real failures happen more often than test failures, for the same test on the same code wi
by peterwwillis 11y ago
Testing doesn't detect failure, it only detects the failure of a test. Real failures happen more often than test failures, for the same test on the same code with the same input and output. The best systematic solution would detect real failures, not see what happens when you fail a test.
- nostrademons 11y agoThat's monitoring, then. As Steve Yegge's Platforms Rant [1] mentioned, testing and monitoring are two sides of the same coin. Google does both, but the original thread-starter here was asking about how to detect failures when the system itself is designed to mask & recover from failures. (FWIW, most such systems do log when they've encountered a failure condition and recovered from it, and this stat is available to the monitoring system.) [1] https://plus.google.com/+RipRowan/posts/eVeouesvaVX https://plus.google.com/+RipRowan/posts/eVeouesvaVX
- peterwwillis 11y agoBasically, yes. But we don't have to make a traditional monitor, or have it be an extra component. Monitoring all the facets of, say, a code deployment, or a software build, or performance testing, is a dynamic thing. It may fail, or it may succeed, or it might be suspicious. Normally we design systems for humans to determine that 3rd part; in this case, there should have been a system where humans could see the one or two pieces of unusual activity and investigated. But there wasn't, or it didn't work right. So a "fix" would be to develop software that adapts to nondeterministic behavior the way a human does. I wouldn't exactly call that monitoring, though.