3 ms·
Is anyone using neural networks for anomaly detection in observability? If so, which model and how many metrics are you supporting per core?
by esafak 3y ago
Is anyone using neural networks for anomaly detection in observability? If so, which model and how many metrics are you supporting per core?
- leeoniya 3y agoLSTM is common for this. also https://facebook.github.io/prophet/ https://facebook.github.io/prophet/
- morkalork 3y agoHow data hungry is it, or what is the minimum volume of data needed before its worth investigating?
- deleted 3y ago[deleted]
- viraptor 3y agoThe more complex the data is, the more you need. If your values are always 5, then you need only one data point.
- morkalork 3y agoIf your values were always 5,you wouldn't use an LSTM to model it either. So presumably there's a threshold for when LSTM becomes practical and useful, no?
- viraptor 3y agoSure, that was an extreme example. The point was that the minimum of data is 1 point, maximum is "all of it". It entirely depends on your use case.
- sarusso 3y agoWhat do you mean by “observability”?
- esafak 3y agoTelemetry. Dashboards. The application is knowing when a signal is anomalous. https://en.wikipedia.org/wiki/Observability_(software) https://en.wikipedia.org/wiki/Observability_(software)
- sarusso 3y agoOh, yes I am working on that. Usually LSTM, exploring encoder-decoders and generative models, but also some simpler models based on periodic averages (which are surprisingly useful in some use cases). But I don’t have per-core metrics.
- tiagod 3y agoDepending on how stable your signal is, I've had good experience with seasonal ARIMA and LOESS (but it's not neural networks)