3 ms·
You can't query this much data with clustering it, indexing it, analyzing it first somehow.
by merinid 13y ago
You can't query this much data with clustering it, indexing it, analyzing it first somehow.
- hga 13y agoNah, check out data warehouses. This data is really limited in details (anything more than from and to telephone numbers, and start and stop times?), and can be shoved into a data warehouse without any analysis as that word is generally understood. Of course it gets indexed, but that's automatic and mindless.
- merinid 13y agoI see your point but I am so tempted to assume that they are not just storing it in some flat dump but trying to extract some sort of higher order representation of the data as they are processing it into the warehouse. You could argue that surfacing trends is technically not "querying".
- merinid 13y agohttp://www.pdl.cmu.edu/SDI/2013/slides/big_graph_nsa_rd_2013_56002v1.pdf http://www.pdl.cmu.edu/SDI/2013/slides/big_graph_nsa_rd_2013...