It's hard to include that information in a meaningful way on a dashboard. It's much easier to alert on that when a metric deviates by a certain percentage. We u

It's hard to include that information in a meaningful way on a dashboard. It's much easier to alert on that when a metric deviates by a certain percentage. We used a service called StatHat that did this automatically, but it could easily be written on top of an in-house data source like Influx or Prometheus. Dashboards are about visualizing how your application works over time, not whether your application is failing. You should know as soon as possible when something has gone wrong and not be waiting until you come in to work in the morning to figure it out. For us, a key stat trending up or down by more than 10% was sent to PagerDuty for someone to investigate. This didn't happen often, but when it did, it was frequently due to something that we were glad we investigated promptly (e.g. abuse/fraud, misbehaving release, etc). It was really rare that we just got that much more or less usage organically.