3 ms·
Monitoring is a tool to help you know when something is already broken, it does nothing to prevent problems, unlike a type system. If your error rates are spiki
by root_axis 7y ago
Monitoring is a tool to help you know when something is already broken, it does nothing to prevent problems, unlike a type system. If your error rates are spiking in production then real customers are getting a bad experience with your product, i.e. hurting sales, your brand image, and generating more work for your colleagues in the form of customer-support tickets that a human still needs to resolve even though you "instantaneously" rolled back the changes.
It's also common to encounter errors in production that are difficult to reproduce in the development environment, this is especially true of data-type errors in dynamic languages where some unusual edge case creates a strange data structure that would simply be impossible with static types.
There's also a chance that the developer fixing the code didn't actually fix it and instead only introduced enough guard logic to handle that one specific case based on their best guess regarding the shape of data that can come through, thus redeploying with a 30% error rate instead of a 60% one, repeating the cycle of attrition testing against the production environment until the error rate is "noise" but maybe not completely fixed.