3 ms·
Additionally, I wonder if any of this fixes the fact that anomaly detection in alerting is traditionally a really hard problem, and one I’ve hardly seen done we
by techpineapple 1y ago
Additionally, I wonder if any of this fixes the fact that anomaly detection in alerting is traditionally a really hard problem, and one I’ve hardly seen done well. Of any set of packaged or recommended alerts, I probably only use 1% of them because anomalies are often the norm.
- tptacek 1y agoThat reinforces his argument.
- techpineapple 1y agoIf he stopped at “LLms can help rca’s, I’d agree with you.
- zer00eyz 1y ago> because anomalies are often the norm You fix these issues or you tune your alert system to make it clear that they aren't actionable. Otherwise you end up turning them off so your system doesn't turn into the boy who cried wolf (or worse teams learn to ignore it and it becomes useless) Bayesian filters, and basic dirivative functions (think math) can do a lot to tame output from these systems. These arent "product features" so in most orgs they dont get the attention they need or deserve.
- zdragnar 1y ago> or worse teams learn to ignore it and it becomes useless This is basically every team I've worked with. Product wants new features, and doesn't want to spend on existing features. Hurry up and write new stuff! Ignore problems and they'll go away! Also: I've already reported this bug! Why haven't the developers fixed it is yet?