2 ms·
> used Spring's AOP to inject performance and usage logs across entire chunks of the program. It worked great and took way less time than scheduled. There was
by SideburnsOfDoom 5y ago
> used Spring's AOP to inject performance and usage logs across entire chunks of the program. It worked great and took way less time than scheduled.
There was the guy who used AOP to inject logging and perf, and set it to "instrument every method".
_Every method_.
It was Ok in test, but when the production load rose to the daily peak, the volume of logging data was in itself sufficient to saturate a network and requests got dropped. Including health checks, which caused machines to be taken out of the load balancer, increasing load on the survivors. Autoscaling didn't help, new machines came up with the same issue.
The failure cascade brought down the entire service.
True story.
- jtolmar 5y agoLuckily for my case, some senior engineer was convinced that it was a good idea to make an "abstraction layer" consisting of pass-throughs for every API and external service, and group all the stuff in that layer in a single package. So I did instrument every method, but only in that BS package.