3 ms·
A Software-based Approach to Identify Heavy Hitters in DNS Traffic
- lima 9y agoWhy not take samples and process that instead? No need to look at all of it.
- teraflop 9y agoTo meet the given sensitivity and precision requirements, you could potentially have to sample a significant fraction of the stream. And then you're still faced with the problem of finding the heavy hitters in the sample.
- makmanalp 9y agoI think it depends on how big you think each thing you want to identify is. If a heavy hitter contains 10% or 50% of traffic, then sampling is absolutely a great approach. Since we're talking about DNS traffic here (very small packets), a "significant" increase in queries for one domain could be e.g. 0.01% or less, which could be much easier to miss with sampling.
- skate22 9y agoCount min sketch guarentees that elements that occur more than a certain number of times (determined by your parameters) will be returned,sampling makes no such promise to a 100% degree of certainty.
- anonacct37 9y agoI find sticky sampling and lossy counting to be useful for this kind of thing. In fact it's what I use to identify heavy hitters of my high traffic DNS servers. http://www.vldb.org/conf/2002/S10P03.pdf http://www.vldb.org/conf/2002/S10P03.pdf
- mwexler 9y agoThis is a good choice, thanks for sharing.
- zimbatm 9y agoIf you haven't already, I recommend getting one of their probes from the Atlas program: https://atlas.ripe.net/landing/get-involved/ https://atlas.ripe.net/landing/get-involved/ It's basically a legal botnet where you can participate by installing one of their probes onto your network. The probe itself is free and you get credits for running it, which you can use to do network measurements like pings to various locations. Did I mention that it's free and really cool?