Monitoring is a pain because we need to monitor more and more complex things. And not only that. Mostly, the problems described by OP are connected to the scale

Monitoring is a pain because we need to monitor more and more complex things. And not only that. Mostly, the problems described by OP are connected to the scale. So the main source of the "pain" is the volume of data we need to process with monitoring systems. That's why Graphite is fading off - it just can't keep up with the growth of data volume. Prometheus is a great tool, but now it is not enough of one server. So you are either looking for workarounds with federation, or running more complex monitoring systems like Thanos, Cortex, VictoriaMetrics.