3 ms·
they'd still need to query their stuff, I guess, so you'd need to trow in there somewhere something to aggregate logs and get the metrics they're tracking out o
by LoSboccacc 9y ago
they'd still need to query their stuff, I guess, so you'd need to trow in there somewhere something to aggregate logs and get the metrics they're tracking out of it - which can totally be done in streaming, without the need of going trough the logs every time, for most metrics.
- hvidgaard 9y agoWith that amount of data, streaming and only saving aggregated data is the only sane way. With 1.2TB/hour there is a limit to how much historical data that can be saved anyway, and we're talking about 30% utilization of a 10gbps network interface, so it's beyond using single machines for most usecases.
- tehlike 9y agoQuery? Most likely not. At least not in the traditional "lets on the fly create a dashboard" sense.
- LoSboccacc 9y agoquery as in 'how much I bill this guy for it's click' - doesn't have to be sql nor on the fly of course