6 ms·

"At least 80% of fibre-optic cables globally go via the US. This is no accident and allows the US to view all communication coming in. At least 80% of all audio

by eli 12y ago

"At least 80% of fibre-optic cables globally go via the US. This is no accident and allows the US to view all communication coming in. At least 80% of all audio calls, not just metadata, are recorded and stored in the US. The NSA lies about what it stores." I think he meant that the NSA had access to the 80% of calls that are routed through the US, not the the NSA is recording and storing literally every single one of them. I think he was misquoted or misspoke. William Binney hasn't worked for the NSA since 2001. Were they recording all calls back then? Did someone still there leak new information to him?

muaddirac 12y ago
"At least 80% of all audio calls, not just metadata, are recorded and stored in the US." It definitely seems like he meant that 80% of calls are recorded and stored. Whether that's true or not will (hopefully) come out in time.
- eli 12y ago
  Sure, if you read just that sentence. If you read just the one before it, it definitely seems like he meant the US has visibility into 80% of calls.
sp332 12y ago
The NSA is recording every call in the Bahamas and Afghanistan. They can listen to any call made in the last 30 days. (Edit: Only cell phone calls, but still.) It's possible that a similar rolling-window system is in place for calls going through the USA. Bahamas: https://firstlook.org/theintercept/article/2014/05/19/data-pirates-caribbean-nsa-recording-every-cell-phone-call-bahamas/ https://firstlook.org/theintercept/article/2014/05/19/data-p... Afghanistan: https://wikileaks.org/WikiLeaks-statement-on-the-mass.html https://wikileaks.org/WikiLeaks-statement-on-the-mass.html
- XorNot 12y ago
  Neither of those two places could be described as having a particularly high volume of calls though. Its an order of magnitude or more difference in sheer volume you'd be looking at.
  - sp332 12y ago
    It's got to be two orders of magnitude easier for them to tap phones in the USA though. They don't have to ingratiate themselves with anyone or hide what they're doing, just show up at AT&T with a FISA-signed warrant and install equipment. https://en.wikipedia.org/wiki/Room_641A https://en.wikipedia.org/wiki/Room_641A They've had a lot longer to do the work, and they have a lot less distance to transport equipment.
    - XorNot 12y ago
      The call in Afghanistan is just satellite calls - those satellites are all US owned. Tapping them is trivial given the very limited number of satellites. US telecommunications is geographically disperse, and my original point anyway was that the volume of calls is enormous - storing and processing that much data is hardly a trivial problem since it's both storage AND CPU intensive. From the US black ops budget you could make some reasonable estimates as to the total size and capability of the NSA to do this.
      - sp332 12y ago
        "The National Security Agency has been recording and storing nearly all the domestic (and international) phone calls from two or more target countries as of 2013." That seems like more than just satellite calls.
      - AlyssaRowan 12y ago
        No. Mobile and landlines, too. Satellite calls have been intercepted worldwide for decades now by numerous intelligence agencies.
  - araes 12y ago
    Well, what would it take to record the whole US? One version says that we make ~5 calls a day on our phones [1] and ownership hovers at 80-90% of the 18+ population (and probably a bunch younger). The US is 311 million, lets say 80% of 300 million, so 240 million people making 5 calls / day. The average call length is 1.8 minutes right now [2]. So we've got 240 mil person * 5 calls / person-day * 365 days * 1.8 min / call. So, about 788 billion minutes of calls / year at a flow rate of about 2 billion calls / day. At MP3 compression of 128kbps, vocal data takes about 0.94 MB / minute [3]. So, close to 688 petabytes of storage data at a rate of about 1.9 petabytes / day. Seems within the realm of doable. The problem of analysing this is in the "ridiculous parallelism" category, so they'd just be constrained by server farm capacity. Lets say they had a system with a conservative million nodes. Each day, each node would have to process ~2 GB of audio data looking for patterns. Not even challenging. If I were clever, I'd probably run a brute force audio to text on each node, then a text to symbol pattern analyser. I'd also have higher level net processes that look for patterns in calls spatially and temporally, but with far less processors. [1] http://www.pewinternet.org/2010/09/02/cell-phones-and-american-adults/ http://www.pewinternet.org/2010/09/02/cell-phones-and-americ... [2] http://www.statista.com/statistics/185828/average-local-mobile-wireless-call-length-in-the-united-states-since-1987/ http://www.statista.com/statistics/185828/average-local-mobi... [3] http://iaudiophile.net/forums/archive/index.php/t-2081.html http://iaudiophile.net/forums/archive/index.php/t-2081.html
fit2rule 12y ago
No: the NSA is recording at least 80% of all calls and will, towards the end of the year, have the capacity to record 100%.
- deleted 12y ago
  [deleted]
- Someone1234 12y ago
  You're assuming that the 20% drop off is due to capacity. That isn't my understanding of the current situation. The Utah facility gives them the ability to store calls for longer, it doesn't increase their scope. The reason they don't record 20% of calls is, from my understanding, that they have internal rules against recording certain people important to the democratic process within the US (e.g. politicians (and their families), judges, election officials, etc). Essentially the politicians were concerned that the NSA's resources would be utilised by whichever party was in office to gain an advantage, so they put themselves (both parties) onto a "no spy" list. But you won't find the law where it says that, because it is all internal-rules created by committee behind closed doors.
  - phkahler 12y ago
    But wouldn't it be nice if they could use their data to get rid of some corruption in politics? I know, I know, how does the government as a whole police itself? Very tough problem. But at some level I don't really care that they collect data, it's what they do with it that's important. So who decides? What is the master agenda? How do they self police? etc... One can only guess at the answers to these questions. It may be nice to allow anyone in the NSA to spy on anyone else in the NSA to prevent abuse, but that would be a huge problem if a spy managed to get a job there. I just imagine this being a rather complex problem.
  - joelhaus 12y ago
    They are not allowed to spy on US persons without a warrant. Full stop. The law is quite clear. Further, the court ruled that metadata was exempt from protection because, in part, it was voluntarily handed over to a third party by the end user. I'd love to hear how you came to your conclusion that only politicians, et. al. enjoy such protections.
Shivetya 12y ago
the techy in me is greatly impressed that can do this. the libertarian in me is greatly annoyed they can do this.
- hnha 12y ago
  It's actually not really something impressive, brewster did some napkin calculation that estimates a cost of about 30 million USD per year. That's nothing. https://blog.archive.org/2013/06/15/cost-to-store-all-us-phonecalls-made-in-a-year-in-cloud-storage-so-it-could-be-datamined/ https://blog.archive.org/2013/06/15/cost-to-store-all-us-pho...
- thenmar 12y ago
  Please don't politicize civil liberties. If progressives and libertarians can come together on anything, it's this.
  - GotAnyMegadeth 12y ago
    Maybe Shivetya doesn't have any progressive in them. If they did, perhaps it would be annoyed too.
  - hnriot 12y ago
    “All cats are libertarians. Completely dependent on others but fully convinced of their own independence.”
    - aridiculous 12y ago
      Good joke, but sort of backfires! Aren't cats notoriously good at surviving in the wild? Both feral and domestic cats (if they have to).
      - wavefunction 12y ago
        Not around here. Cats are great meals for coyotes, foxes, birds of prey, feral hogs... The list goes on and on. But don't tell a cat that and burst their bubble!
nitrogen 12y ago
Back then the key words were "echelon" and "carnivore". People had suspicions, IIRC there was an EU inquiry, but only now do we have concrete evidence.
AlyssaRowan 12y ago
Very probably yes. GCHQ's retention goes back (very broadly) that far (it's inconsistent) - and they have some access to each other's networks (the NSA access GCHQ's network via example.gchq.nsa.ic.gov, specifically, so for example https://wiki.gchq.nsa.ic.gov/index.php https://wiki.gchq.nsa.ic.gov/index.php would get you to GCHQ's internal MediaWiki if you're on the NSA network - GCHQ use an internal .gchq TLD). They can query selectors on each other's databases, and filtering of what's "not allowed" seems to be very often up to the analyst. (Of course, I don't know if they share everything. Probably not.) The NSA's own BLARNEY initiative for this stuff dates back to 1978, although I think GCHQ beat them to the punch in effective mass telephony intercept - the first one I'm aware of started in the early 1980s, although that is offline, and probably no longer archived as the media (digital tape with low-bitrate encoded audio) would probably have degraded beyond all usefulness by now (and it sounded pretty crunchy in the first place, I understand, kinda like a bad squelchy lower-side-band transmission!). Remember, however, that they didn't have the same kind of analysis capability back then that they've got these days (but they can probably go back and analyse old stuff they still retain). As Snowden's disclosed, you're seeing the newer systems having full-take ring buffers in nearline storage, followed by offline selection for recent access and analysis from that using a huge amount of distributed processing, and in turn automatic selected archiving out of that. It's pretty much the difference between microfiche and Google in effectiveness.
DennisP 12y ago
I have no idea what the NSA stores, but it's certainly feasible to store all the audio calls if they want to, and the new data center they're building has far more capacity than they would need: http://blog.archive.org/2013/06/15/cost-to-store-all-us-phonecalls-made-in-a-year-in-cloud-storage-so-it-could-be-datamined/ http://blog.archive.org/2013/06/15/cost-to-store-all-us-phon...
asgard1024 12y ago
I don't think there is any doubt about the fact that NSA wants to store it all. Just look at the short interview/documentary with Binney that was linked from the article: https://www.youtube.com/watch?v=590cy1biewc https://www.youtube.com/watch?v=590cy1biewc

by eli 12y ago

muaddirac 12y ago

"At least 80% of all audio calls, not just metadata, are recorded and stored in the US." It definitely seems like he meant that 80% of calls are recorded and stored. Whether that's true or not will (hopefully) come out in time.

eli 12y ago
Sure, if you read just that sentence. If you read just the one before it, it definitely seems like he meant the US has visibility into 80% of calls.

sp332 12y ago

The NSA is recording every call in the Bahamas and Afghanistan. They can listen to any call made in the last 30 days. (Edit: Only cell phone calls, but still.) It's possible that a similar rolling-window system is in place for calls going through the USA. Bahamas: https://firstlook.org/theintercept/article/2014/05/19/data-pirates-caribbean-nsa-recording-every-cell-phone-call-bahamas/ https://firstlook.org/theintercept/article/2014/05/19/data-p... Afghanistan: https://wikileaks.org/WikiLeaks-statement-on-the-mass.html https://wikileaks.org/WikiLeaks-statement-on-the-mass.html

XorNot 12y ago
Neither of those two places could be described as having a particularly high volume of calls though. Its an order of magnitude or more difference in sheer volume you'd be looking at.
- sp332 12y ago
  It's got to be two orders of magnitude easier for them to tap phones in the USA though. They don't have to ingratiate themselves with anyone or hide what they're doing, just show up at AT&T with a FISA-signed warrant and install equipment. https://en.wikipedia.org/wiki/Room_641A https://en.wikipedia.org/wiki/Room_641A They've had a lot longer to do the work, and they have a lot less distance to transport equipment.
  - XorNot 12y ago
    The call in Afghanistan is just satellite calls - those satellites are all US owned. Tapping them is trivial given the very limited number of satellites. US telecommunications is geographically disperse, and my original point anyway was that the volume of calls is enormous - storing and processing that much data is hardly a trivial problem since it's both storage AND CPU intensive. From the US black ops budget you could make some reasonable estimates as to the total size and capability of the NSA to do this.
    - sp332 12y ago
      "The National Security Agency has been recording and storing nearly all the domestic (and international) phone calls from two or more target countries as of 2013." That seems like more than just satellite calls.
    - AlyssaRowan 12y ago
      No. Mobile and landlines, too. Satellite calls have been intercepted worldwide for decades now by numerous intelligence agencies.
- araes 12y ago
  Well, what would it take to record the whole US? One version says that we make ~5 calls a day on our phones [1] and ownership hovers at 80-90% of the 18+ population (and probably a bunch younger). The US is 311 million, lets say 80% of 300 million, so 240 million people making 5 calls / day. The average call length is 1.8 minutes right now [2]. So we've got 240 mil person * 5 calls / person-day * 365 days * 1.8 min / call. So, about 788 billion minutes of calls / year at a flow rate of about 2 billion calls / day. At MP3 compression of 128kbps, vocal data takes about 0.94 MB / minute [3]. So, close to 688 petabytes of storage data at a rate of about 1.9 petabytes / day. Seems within the realm of doable. The problem of analysing this is in the "ridiculous parallelism" category, so they'd just be constrained by server farm capacity. Lets say they had a system with a conservative million nodes. Each day, each node would have to process ~2 GB of audio data looking for patterns. Not even challenging. If I were clever, I'd probably run a brute force audio to text on each node, then a text to symbol pattern analyser. I'd also have higher level net processes that look for patterns in calls spatially and temporally, but with far less processors. [1] http://www.pewinternet.org/2010/09/02/cell-phones-and-american-adults/ http://www.pewinternet.org/2010/09/02/cell-phones-and-americ... [2] http://www.statista.com/statistics/185828/average-local-mobile-wireless-call-length-in-the-united-states-since-1987/ http://www.statista.com/statistics/185828/average-local-mobi... [3] http://iaudiophile.net/forums/archive/index.php/t-2081.html http://iaudiophile.net/forums/archive/index.php/t-2081.html

fit2rule 12y ago

No: the NSA is recording at least 80% of all calls and will, towards the end of the year, have the capacity to record 100%.

deleted 12y ago
[deleted]
Someone1234 12y ago
You're assuming that the 20% drop off is due to capacity. That isn't my understanding of the current situation. The Utah facility gives them the ability to store calls for longer, it doesn't increase their scope. The reason they don't record 20% of calls is, from my understanding, that they have internal rules against recording certain people important to the democratic process within the US (e.g. politicians (and their families), judges, election officials, etc). Essentially the politicians were concerned that the NSA's resources would be utilised by whichever party was in office to gain an advantage, so they put themselves (both parties) onto a "no spy" list. But you won't find the law where it says that, because it is all internal-rules created by committee behind closed doors.
- phkahler 12y ago
  But wouldn't it be nice if they could use their data to get rid of some corruption in politics? I know, I know, how does the government as a whole police itself? Very tough problem. But at some level I don't really care that they collect data, it's what they do with it that's important. So who decides? What is the master agenda? How do they self police? etc... One can only guess at the answers to these questions. It may be nice to allow anyone in the NSA to spy on anyone else in the NSA to prevent abuse, but that would be a huge problem if a spy managed to get a job there. I just imagine this being a rather complex problem.
- joelhaus 12y ago
  They are not allowed to spy on US persons without a warrant. Full stop. The law is quite clear. Further, the court ruled that metadata was exempt from protection because, in part, it was voluntarily handed over to a third party by the end user. I'd love to hear how you came to your conclusion that only politicians, et. al. enjoy such protections.

Shivetya 12y ago

the techy in me is greatly impressed that can do this. the libertarian in me is greatly annoyed they can do this.

hnha 12y ago
It's actually not really something impressive, brewster did some napkin calculation that estimates a cost of about 30 million USD per year. That's nothing. https://blog.archive.org/2013/06/15/cost-to-store-all-us-phonecalls-made-in-a-year-in-cloud-storage-so-it-could-be-datamined/ https://blog.archive.org/2013/06/15/cost-to-store-all-us-pho...
thenmar 12y ago
Please don't politicize civil liberties. If progressives and libertarians can come together on anything, it's this.
- GotAnyMegadeth 12y ago
  Maybe Shivetya doesn't have any progressive in them. If they did, perhaps it would be annoyed too.
- hnriot 12y ago
  “All cats are libertarians. Completely dependent on others but fully convinced of their own independence.”
  - aridiculous 12y ago
    Good joke, but sort of backfires! Aren't cats notoriously good at surviving in the wild? Both feral and domestic cats (if they have to).
    - wavefunction 12y ago
      Not around here. Cats are great meals for coyotes, foxes, birds of prey, feral hogs... The list goes on and on. But don't tell a cat that and burst their bubble!

nitrogen 12y ago

Back then the key words were "echelon" and "carnivore". People had suspicions, IIRC there was an EU inquiry, but only now do we have concrete evidence.

AlyssaRowan 12y ago

Very probably yes. GCHQ's retention goes back (very broadly) that far (it's inconsistent) - and they have some access to each other's networks (the NSA access GCHQ's network via example.gchq.nsa.ic.gov, specifically, so for example https://wiki.gchq.nsa.ic.gov/index.php https://wiki.gchq.nsa.ic.gov/index.php would get you to GCHQ's internal MediaWiki if you're on the NSA network - GCHQ use an internal .gchq TLD). They can query selectors on each other's databases, and filtering of what's "not allowed" seems to be very often up to the analyst. (Of course, I don't know if they share everything. Probably not.) The NSA's own BLARNEY initiative for this stuff dates back to 1978, although I think GCHQ beat them to the punch in effective mass telephony intercept - the first one I'm aware of started in the early 1980s, although that is offline, and probably no longer archived as the media (digital tape with low-bitrate encoded audio) would probably have degraded beyond all usefulness by now (and it sounded pretty crunchy in the first place, I understand, kinda like a bad squelchy lower-side-band transmission!). Remember, however, that they didn't have the same kind of analysis capability back then that they've got these days (but they can probably go back and analyse old stuff they still retain). As Snowden's disclosed, you're seeing the newer systems having full-take ring buffers in nearline storage, followed by offline selection for recent access and analysis from that using a huge amount of distributed processing, and in turn automatic selected archiving out of that. It's pretty much the difference between microfiche and Google in effectiveness.

DennisP 12y ago

I have no idea what the NSA stores, but it's certainly feasible to store all the audio calls if they want to, and the new data center they're building has far more capacity than they would need: http://blog.archive.org/2013/06/15/cost-to-store-all-us-phonecalls-made-in-a-year-in-cloud-storage-so-it-could-be-datamined/ http://blog.archive.org/2013/06/15/cost-to-store-all-us-phon...

asgard1024 12y ago

I don't think there is any doubt about the fact that NSA wants to store it all. Just look at the short interview/documentary with Binney that was linked from the article: https://www.youtube.com/watch?v=590cy1biewc https://www.youtube.com/watch?v=590cy1biewc