6 ms·
Did I miss an elephant in the room? Wide structured logging to log EVERYTHING? Isn't that just massively huge? I don't see how that would be cheaper. Related
by tunesmith 2y ago
Did I miss an elephant in the room?
Wide structured logging to log EVERYTHING? Isn't that just massively huge? I don't see how that would be cheaper.
Related Steven Wright joke: “I have a map of the United States... Actual size. It says, 'Scale: 1 mile = 1 mile.' I spent last summer folding it. I hardly ever unroll it. People ask me where I live, and I say, 'E6.”
- phillipcarter 2y ago> I don't see how that would be cheaper. It's cheaper for several tools that bill by number of events rather than total volume of data in GB. The way this works with very high volumes of data is to employ smarter sampling to make sure you get as good a ratio of good vs. useless events as possible within a given budget. Observability in this fashion is much more like real-time analytics (with an appropriate backend, i.e., not a timeseries database), where the cost of querying an event that as 2 fields compared to 200 fields is marginal. And so in a world like this, you're encouraged to pack more information into each log/event/span. There's a lot of details underlying that, like some backends still requiring you to define a subset you'd like to always be able to group by, whereas other backends have no such limitations, but this is largely the category of system that's being talked about.