> unlikely that anybody, including the NSA, can collect, index, catalog and parse meaning out of hundreds of GB _per second No, it's actually very likely. I've

> unlikely that anybody, including the NSA, can collect, index, catalog and parse meaning out of hundreds of GB _per second No, it's actually very likely. I've noticed people often make this argument as if it is obvious that the data that is being processed is somehow too large to handle. But they misunderstand the ways this can be done and underestimate current technological capabilities. One big switch in single datacenter can handle all traffic of that datacenter, hundreds of GB/s. Make a special one with custom FPGA/cpus and in addition to switching, it can do decrypting, parsing, matching and filtering the way you like, hundreds of GB per second, real-time. This is a highly paralelizable job and the hardware is made to exploit that. The resulting stream of interesting data is probably much thinner and can be easily sent to some central location, realtime. The rest can be stored locally and sent on demand.