3 ms·
To your first point, I agree that's probably a smart allocation to make given the expense. Where we differ is the value driven by lower order tests; synthetic t
by kodah 3y ago
To your first point, I agree that's probably a smart allocation to make given the expense. Where we differ is the value driven by lower order tests; synthetic test breakages are also more expensive fixes since they've traveled the SDLC. Overweighting synthetic tests compared to unit and integration can lead to a lack of confidence in proposed changes. That's to say, I think it's good to have the larger volume of your testing allocated in unit and integration tests while synthetic tests can validate major contracts.
> If a synthetic test breaks production in some unanticipated way, that is incredibly valuable because one user shouldn't be able to break production for everyone and you just found one heck of a bug to address.
This is a valid point, however, I wasn't referring to breakages. I was referring to usage statistics, data, etc. Conflating synthetic usage and data with real usage and data can be problematic. There's ways to mitigate that, but they add overhead, which was my original point that synthetic testing and monitoring adds a good deal of operational and code complexity.