Can we please stop obsessively monitoring (and $deity forbid, alerting on) low level metrics such as CPU and Disk I/O? Sure, they should be recorded as they are

Can we please stop obsessively monitoring (and $deity forbid, alerting on) low level metrics such as CPU and Disk I/O? Sure, they should be recorded as they are useful for troubleshooting and looking for bottlenecks. But relying on them for most software is trying to figure out if you should get a speeding ticket by looking at the tachometer. CPU usage being high doesn't really tell you anything. You may suspect that there's an issue if you are familiar with the system, and you have only one system to worry about. It's definitely not OK if you have a 'modern' architecture and a fleet of machines running a distributed system. There's a whole lot of noise and you can't tell if traffic has increased, if someone pushed inefficient code, if there's a cooling problem and CPU is throttled, if a "batch" process started, or a million of reasons. And even if it is high, and it is an anomaly(anomaly detection on those metrics can be useful at times), is it causing any issues for your customers? CPU usage can't tell you that. Every piece of software has some work to do. What is that work? Figure that out and monitor that. Golden signals on the relevant metrics and you'll be in a far better position. Did the error rate increase? Are we getting higher response times? In that case it doesn't matter what the CPU metric says – it could even be lower than usual if there's a bottleneck somewhere else – it is a problem that needs to be addressed. You can then use the other metrics to confirm, or try to isolate the problem. But they should be supporting information, not your main diagnostics tool. > In a project I worked on, our development builds started writing about 80 MB of log messages per second to disk. Well, there you go. If you just look at disk I/O you'll see that it is high. But if you track metrics from your logging system, you'll be able to immediately see _why_ Datadog (no affiliation here) has a good blog that set me on the right track: https://www.datadoghq.com/blog/monitoring-101-collecting-data/ https://www.datadoghq.com/blog/monitoring-101-collecting-dat... By all means monitor those metrics on your work laptop - very few people are going to bother running prometheus in their dev machine and setting up alerts for third party apps. But for the apps you are responsible for? Figure out, from day zero, what metrics you should collect for your app and monitor those instead.