I'd say retaining logs is the real mistake. You need to retain some, but it's going to be far less then you think. The more interesting data is usually "what's

I'd say retaining logs is the real mistake. You need to retain some, but it's going to be far less then you think. The more interesting data is usually "what's being logged right now" and none of the big logging systems have an answer as good as "tail -f somefile.log" or "journalctl -f -u some-service" on the machine it's running on...which sucks, because technically there's value to be found there! Connect me to the entire cluster I'm running and tail "just these logs" in as close to real-time as you can. EDIT: I know it can be emulated in a bunch of ways, but it's certainly not a first class feature.