3 ms·
There's a wide variety of time series data that is interesting to track: metrics, application telemetry, kpis, auditing events, log messages, etc. This is rich
by jillesvangurp 7y ago
There's a wide variety of time series data that is interesting to track: metrics, application telemetry, kpis, auditing events, log messages, etc. This is rich structured data. The richer it is; the more useful it gets. Many companies use different solutions for each of those and that creates a lot of complexity and devops overhead. I've seen a lot of projects where corners get cut and there is just the bare minimum of data. With microservices and serverless, the problem is even more acute because logging into individual servers and grepping through logs like a caveman is just not practical anymore. Running blind is downright irresponsible and sadly all too common.
When something happens, it's actually interesting cross examine all of these different things. When you do, having meta data to drill down and break down is essential and it's also essential to have software that can handle that that doesn't fall over just because you had a spike in usage. Having everything in one place and annotated with the right meta data is key. Another thing that is key is retention policies to keep the data volume in check. Done properly, you will be generating many GB in data per day. Left unchecked, this will kill whatever infrastructure you have in no time. And while historical data is sometimes interesting, it's usually the very recent past that is most interesting. So, you should expect to throw away the vast majority of data you collect after a short while.