4 ms·
The most important thing that was missed: Good: Alerts on business level metrics Bad: Alerts on machine level metrics Knowing that checkout volume is sharply
by jedberg 5y ago
The most important thing that was missed:
Good: Alerts on business level metrics
Bad: Alerts on machine level metrics
Knowing that checkout volume is sharply down is far more valuable than knowing that CPU on one of the checkout servers is way up. Mainly because that high CPU may have no customer effect, so it's really not all that urgent.
- Phelinofist 5y agoWhy not both? Scraping machine level metrics works out of the box with most agents
- jedberg 5y agoIt's fine to scrape the metrics, but what I'm saying is don't alert on them by default until you are sure that a particular server metric is actually a good alert.
- cassianoleal 5y ago> Good: Alerts on business level metrics I agree partially. I would make it more general though: alert on symptoms of problems. Those can be business metrics, like the one you suggested in your example, or they can be system level, like rate of errors, or a queue that's growing out of control. > Bad: Alerts on machine level metrics 100% agree. There is no excuse for that. A CPU working overtime with no customer impact (no symptoms of problems) is an efficient system. I'm paying for that CPU, I'd like to use it. If I get an alert every time I use something I pay for that will only drive me to pay more so it shuts up - even though there was no problem to begin with.
- lxe 5y agoYes. And also make business metrics somehow traceable to machine metrics.