4 ms·
I recently did the following: - had a giant pcap - wrote a perl script to output some of the key value from the dump (e.g. IP and UDP packet lengths) into csv
by alexpotato 4y ago
I recently did the following:
- had a giant pcap
- wrote a perl script to output some of the key value from the dump (e.g. IP and UDP packet lengths) into csv
- loaded the csv into sqlite3 database
- ran several queries to identify microbursts of bandwidth etc
The younger/more junior folks were blown away that you could do this with <100 lines of code and it was pretty fast.
Btw, above was inspired by this: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...
- bqmjjx0kac 4y agoIf I were looking for bursts, SQL is not the first thing that comes to mind! Could you elaborate on this or sketch out the query?
- alexpotato 4y agoBasically, doing a group by at millisecond resolution with a sum on the IP packet length to get a rough metric for bandwidth. Once you have that, you can see the milliseconds with the highest bandwidth. Some extra math can also get you to Gigabits/second in a more network engineer friendly format.
- bqmjjx0kac 4y agoAha, group by millisecond! Thanks.
- anitil 4y agoI did a histogram-type thing in the same way by using a window function (similarly sqlite table scraped off pcap recordings). I can't remember if it was a fixed-width window (number of samples) or within some time window Dropped it in datasette with datasette-vega and got a nice little plot