4 ms·

I don't know what the media coverage stated, but the source material indicates that full-take is only stored in a 5 day rolling buffer. I believe 30-day for cer

I don't know what the media coverage stated, but the source material indicates that full-take is only stored in a 5 day rolling buffer. I believe 30-day for certain flagged selectors. Not at all indefinite, that is not possible.

hendersoon 10y ago
As of 2011, GCHQ stored their full-take for 3 days and metadata for 30 days. Do you believe they haven't increased capacity in the past 6 years? Again, the discussion comes down to evidence versus reasoned speculation. Evidence of intelligence agency activity is generally pretty tough to come by.
- Karunamon 10y ago
  I'd also believe the country's data transmission volume has increased in the same timeframe, and given how IOT is rapidly scaling up, probably a lot faster than storage capacity.
  - hendersoon 10y ago
    They have techniques to reduce the data corpus, called "massive volume reduction" or MVR in the GCHQ "Mastering the Internet" rather pompously named leak. It discards junk traffic and assumedly does deduplication, etc. Remember, Tempora went online in 2011, 6 years ago. Techniques have certainly been refined since. [1] https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secret-world-communications-nsa https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secre... [2] https://www.theguardian.com/uk/2013/jun/21/gchq-mastering-the-internet https://www.theguardian.com/uk/2013/jun/21/gchq-mastering-th...
    - Karunamon 10y ago
      But we're talking encrypted streams here. No algorithm, real or imagined, can dedupe or compress random noise.
- willstrafach 10y ago
  Additional data intake would greatly increase costs as well. I guess I'm not fully understanding, is the implication that getting more data would lead to more useful information? From reading the source documentation itself (not editorialized coverage), it seems they are already swimming in almost too much data. Their focus is much more likely to be aimed at data refinement at this point, as adding even more data would make it far more difficult for analysts to do their job.
  - hendersoon 10y ago
    It absolutely will NOT be more useful! That's one of the main takeaways from Snowden's leaks-- mass surveillance does not make us safer. Thing is, the three-letter agencies strongly disagree.

hendersoon 10y ago

As of 2011, GCHQ stored their full-take for 3 days and metadata for 30 days. Do you believe they haven't increased capacity in the past 6 years? Again, the discussion comes down to evidence versus reasoned speculation. Evidence of intelligence agency activity is generally pretty tough to come by.

Karunamon 10y ago
I'd also believe the country's data transmission volume has increased in the same timeframe, and given how IOT is rapidly scaling up, probably a lot faster than storage capacity.
- hendersoon 10y ago
  They have techniques to reduce the data corpus, called "massive volume reduction" or MVR in the GCHQ "Mastering the Internet" rather pompously named leak. It discards junk traffic and assumedly does deduplication, etc. Remember, Tempora went online in 2011, 6 years ago. Techniques have certainly been refined since. [1] https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secret-world-communications-nsa https://www.theguardian.com/uk/2013/jun/21/gchq-cables-secre... [2] https://www.theguardian.com/uk/2013/jun/21/gchq-mastering-the-internet https://www.theguardian.com/uk/2013/jun/21/gchq-mastering-th...
  - Karunamon 10y ago
    But we're talking encrypted streams here. No algorithm, real or imagined, can dedupe or compress random noise.
willstrafach 10y ago
Additional data intake would greatly increase costs as well. I guess I'm not fully understanding, is the implication that getting more data would lead to more useful information? From reading the source documentation itself (not editorialized coverage), it seems they are already swimming in almost too much data. Their focus is much more likely to be aimed at data refinement at this point, as adding even more data would make it far more difficult for analysts to do their job.
- hendersoon 10y ago
  It absolutely will NOT be more useful! That's one of the main takeaways from Snowden's leaks-- mass surveillance does not make us safer. Thing is, the three-letter agencies strongly disagree.