Having a corpus of live "test" requests for rebuilding an API is a compelling enough use case to justify infinite retention, in my opinion. Perhaps relevantly t

Having a corpus of live "test" requests for rebuilding an API is a compelling enough use case to justify infinite retention, in my opinion. Perhaps relevantly to your prior gig: "did we handle that weird Sony Bravia API call from five years ago in this redesign? Yes we did." I'm implying a little more structure to logging, though, because it sounds like you're thinking about random strings emitted from binaries (which I think are also approachable, to an extent). This overlaps with PII and such, though, and there is no one-size-fits-all answer. More than once I've run the entire corpus of requests to a system, ever, through a dummy rebuild as a pretty great integration test. It's a powerful SRE tool. Spelunking through all historical data is just icing on that cake, honestly. As the author says, SRE is basically just an information factory; I'd be haaaaaard pressed to agree with you on throwing away a lot of information -- you don't know what you don't know until you want to know it -- and betting all-in on monitoring. Retaining logs is not the hardest problem SRE deals with, either, but SREs turn around and force unrealistic latency requirements on the query side (I see a lot of ELK deploys running into this). You have to look at it as an Oracle. Oh, great Oracle of a pile of meaningless logs, cook off this map/reduce and tell me an interesting number that I can put in a Keynote for executives. Definitely not dashboarding from logs data. That's an impedance mismatch that Google gets away with because of the nature of their logging.