4 ms·

What kind of software is used to do this?

by bprater 19y ago

tptacek 19y ago
The leaders in the space are Sandvine and Cisco, and Arbor (my old employer) just bought their way in by nabbing Ellacoya. The answer in the general case is, "stuff that looks substantially like libpcap". It's all written in C, surprise! On the very high end, or in very specialized cases, you'll find: * FPGA regexers, OEM'd from a couple common vendors, that compile DFAs into gates. * Multicore "network processors" with MIPS cores and fast custom memory busses. * Blades that connect into the backplanes of (ubiquitous) Cat6k switches to intercept traffic. By and large, this is a systems design problem, not an algorithms challenge. To appreciate that, you have to get the context for this WaPo story, which includes the fact that the story is totally overblown and ISPs are not reading your email or web traffic, but rather trying to figure out how to commercialize a "fast-path" network product for bulk P2P/streaming media customers.
- anupamkapoor 19y ago
  shouldn't narus-networks be up there somewhere ? edit: also, packet classification is quite cool from research perspective. for example, check out the following : "packet classification on multiple fields, gupta, mckewon". also, network algorithmics devotes an entire chapter on various strategies for efficient packet-classfication.
  - tptacek 19y ago
    Network Algorithmics is an awesome book; it's been recommended here before. But I don't think most of the DPI products use advanced packet classification algorithms. No product I've worked on has; it's pretty much, "that's port 80, so use the HTTP decode". There's classification done for binning and accounting, but it's pretty brute force. Narus is a DPI vendor, but not of the type WaPo is talking about. They provide "lawful" (read, "unlawful") intercept for traffic that has already been classified and diverted.

4 ms·

What kind of software is used to do this?

by bprater 19y ago

What kind of software is used to do this?

tptacek 19y ago
The leaders in the space are Sandvine and Cisco, and Arbor (my old employer) just bought their way in by nabbing Ellacoya. The answer in the general case is, "stuff that looks substantially like libpcap". It's all written in C, surprise! On the very high end, or in very specialized cases, you'll find: * FPGA regexers, OEM'd from a couple common vendors, that compile DFAs into gates. * Multicore "network processors" with MIPS cores and fast custom memory busses. * Blades that connect into the backplanes of (ubiquitous) Cat6k switches to intercept traffic. By and large, this is a systems design problem, not an algorithms challenge. To appreciate that, you have to get the context for this WaPo story, which includes the fact that the story is totally overblown and ISPs are not reading your email or web traffic, but rather trying to figure out how to commercialize a "fast-path" network product for bulk P2P/streaming media customers.
- anupamkapoor 19y ago
  shouldn't narus-networks be up there somewhere ? edit: also, packet classification is quite cool from research perspective. for example, check out the following : "packet classification on multiple fields, gupta, mckewon". also, network algorithmics devotes an entire chapter on various strategies for efficient packet-classfication.
  - tptacek 19y ago
    Network Algorithmics is an awesome book; it's been recommended here before. But I don't think most of the DPI products use advanced packet classification algorithms. No product I've worked on has; it's pretty much, "that's port 80, so use the HTTP decode". There's classification done for binning and accounting, but it's pretty brute force. Narus is a DPI vendor, but not of the type WaPo is talking about. They provide "lawful" (read, "unlawful") intercept for traffic that has already been classified and diverted.