4 ms·

I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques. Also I doubt the veracity

I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques. Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.

quasque 13y ago
Another benefit could be realised in the future if historians and linguists manage to get access to all this data. I imagine that researchers of the social sciences would end up enjoying the same sort of large-scale collaborative projects that particle physicists or genomic bioinformaticians currently have with their huge datasets.
nikcub 13y ago
Hadoop[0] and OpenStack[1] I think the era of government being far ahead of commercial tech capability is over. The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. [0] http://online.wsj.com/article/SB10001424127887323495604578535290627442964.html http://online.wsj.com/article/SB1000142412788732349560457853... [1] http://www.youtube.com/watch?v=NgahKksMZis http://www.youtube.com/watch?v=NgahKksMZis
- 286c8cb04bda 13y ago
  The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. There are a lot of fingers in that pie. Oracle, for example, has a National Security Group, whose job is to come up with "solutions" and then try to sell them to three-letter-agencies.
coldtea 13y ago
>Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? Why would they? They deal with ISPs, backbones and such.
aryastark 13y ago
> I wonder how they store all that. Well, now they have a massive data center in Utah. That's most likely where it's all going today. Standard open source tools are all you really need. > Wouldn't they show up on, say, Sandvine's Internet traffic reports? No. If some script kiddie/hacker type installs a packet sniffer and logs all your traffic to your ISP, that won't show up anywhere. Traffic goes somewhere. You're sending packets out. Merely logging packets is entirely passive and undetectable.

quasque 13y ago

Another benefit could be realised in the future if historians and linguists manage to get access to all this data. I imagine that researchers of the social sciences would end up enjoying the same sort of large-scale collaborative projects that particle physicists or genomic bioinformaticians currently have with their huge datasets.

nikcub 13y ago

Hadoop[0] and OpenStack[1] I think the era of government being far ahead of commercial tech capability is over. The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. [0] http://online.wsj.com/article/SB10001424127887323495604578535290627442964.html http://online.wsj.com/article/SB1000142412788732349560457853... [1] http://www.youtube.com/watch?v=NgahKksMZis http://www.youtube.com/watch?v=NgahKksMZis

286c8cb04bda 13y ago
The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors. There are a lot of fingers in that pie. Oracle, for example, has a National Security Group, whose job is to come up with "solutions" and then try to sell them to three-letter-agencies.

coldtea 13y ago

>Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? Why would they? They deal with ISPs, backbones and such.

aryastark 13y ago

> I wonder how they store all that. Well, now they have a massive data center in Utah. That's most likely where it's all going today. Standard open source tools are all you really need. > Wouldn't they show up on, say, Sandvine's Internet traffic reports? No. If some script kiddie/hacker type installs a packet sniffer and logs all your traffic to your ISP, that won't show up anywhere. Traffic goes somewhere. You're sending packets out. Merely logging packets is entirely passive and undetectable.