5 ms·

You might as well throw out any number. The base here is developed from the author's "family average". That doesn't, in any way, reflect "all US phonecalls". C

You might as well throw out any number. The base here is developed from the author's "family average". That doesn't, in any way, reflect "all US phonecalls". Consider business users. There are a substantial number of business users who talk on the phone for >1,000 minutes per month. "Family" averages are only going to reflect personal phone calls, which are a fraction of the phone calls made. We also cannot assume equivalency between what the Internet Archive pays per petabyte and what the NSA pays per petabyte. When dealing with government projects, you have all manner of requirements that have no parallel in the rest of the business world. That $27.2 million number might as well be $50 million, or $100 million. It all depends on your input variables. This is napkin math at its worst.

maresca 13y ago
The point is that even quadrupled, the number is so low, it's a drop in the bucket for the government.
tommi 13y ago
Napkin math sure, but not at its worst. It's just an very rough estimation with explanations for numbers given and not claiming to be exact. I think the most valuable lesson here is that it's likely to be in the ballpark of millions of dollars and that in theory its possible to do it for $27.2 million.
arscan 13y ago
I think the point here is that its not some ridiculously huge number, like $100 billion. It is feasible for the NSA to actually store a substantial percentage of US phone calls if they wanted.
rm999 13y ago
It's safe to assume these kinds of estimates are "order of magnitude" estimates, meaning they could be off by a factor of 2 or more. The goal isn't to build your own system or to bill the government, it's to put the numbers in perspective. Case in point: if you had asked me if phone recording cost closer to 10 billion dollars or 100 million dollars I would have guessed 10 billion; now I know better. Being able to quickly do order of magnitude estimates (especially in your head) is a very important skill IMO. I've heard McKinsey makes it part of their interview process.
- larrys 13y ago
  Also need to tack on the cost of developing the system, oversight, bidding etc. Still I would have guess much higher as well.
ctdonath 13y ago
Napkin math still gets a ballpark notion. Let's compute an outer limit: record everyone, all the time, CD quality. 44100 samples per second * 2 bytes per sample * 2 channels * 60 seconds per minute * 60 minutes per hour * 24 hours per day * 365 days per year * 313900000 people * $100 per terabyte / 1 terabyte = $175 billion per year. That's an absolute outer limit for cost. That's less than 5% of federal budget, and we haven't started on the obvious ways to cut costs by several orders of magnitude. Reduce it to 4410 samples/sec, 1 byte per sample, 1 channel, 1/10th the time and we're already under $0.5B/yr, without even addressing audio compression (much less voice-to-text).
- bradleyland 13y ago
  So what inference are we drawing? I honestly wasn't aware that the capability to store all voice traffic was in question. That might be owed to my background in telecom though. Telephone codecs are extremely low bitrate (relative to something like music, much less video) because you have very clear design constraints, and those constraints are forgiving. You just need to be able to understand the caller's voice, not accurately reproduce a live performance of Beethoven's 5th. I agree that the estimates to store the data are on the very, very high side, but I'm not sure that number is even significant in the whole scope of the challenge. The challenges in obtaining every phone call made in the US aren't storage related; they're almost entirely collection and aggregation related. These ballpark numbers don't even attempt to factor in that portion of the cost. IMO, the collection, transport, and aggregation systems are easily 8/10ths of the problem, not storage. Storage is an extremely scalable solution. If the government were going to do something like this, they'd likely tap in to the phone networks at the same places everyone else does. There are major telecom aggregation points - called "tandems" [1] - around the country. If you want to set up your own phone carrier with an actual physical network of your own, this is where you plug in. The government would have to do the same. They can't simply tap in at a single aggregation point, because not all phone calls pass through the same points. From there, you face the choice of storing the data regionally in several data centers, or attempting to aggregate it all back to a central data center. IMO, a disaggregated approach makes a lot more sense. You could reasonably expect to transport all CDR [2] data back to a central location, but you wouldn't want to send all media (audio data) back to one place. It would be simple enough to only fetch what you need based on a query against call data. You'd want all the the CDR data in one place so you could perform your "big data" analysis on it efficiently, then cherry pick media to pull in for analysis. I'm certain I haven't even scratched the surface here. This only gets the government long distance calls. It doesn't touch local, or even intraLATA calls. Maybe the government isn't interested in those calls though. Maybe they're only interested in international calls, which makes the problem simpler, not harder. Others in the thread have mentioned that this should be treated as an "order of magnitude" estimation. I don't think we can draw any inference from this estimation at all, because it represents such a small portion of the problem domain. We also have no idea what the scope of the challenge is. The entire exercise is pointless when you think about what we're asking. "Can the government actually implement a solution to record every phone call in the US?" I think that's a resounding yes. The solution would look a lot like setting up a tier 2 network provider [3] with extra investment in a storage back end. That's entirely within the realm of possibility given the size of the US Dept of Defense budget. [1] http://en.wikipedia.org/wiki/Class_4_telephone_switch http://en.wikipedia.org/wiki/Class_4_telephone_switch [2] http://en.wikipedia.org/wiki/Call_detail_record http://en.wikipedia.org/wiki/Call_detail_record [3] http://en.wikipedia.org/wiki/Tier_2_network http://en.wikipedia.org/wiki/Tier_2_network
dagw 13y ago
Sure, but the point doesn't change. All the calls made in the US can be stored on a budget that isn't out of reach for an organization like the NSA. Napkin math is all about getting you within the right order of magnitude and this manages that just fine.
skue 13y ago
OTOH, the estimated bytes per call truly is conservative. The 8KB/s estimate is based on G.711, and using G.726 or G.729 would yield a 2-8x reduction in storage. So this is definitely within an order of magnitude: even if the minutes of talk time are off by a factor of 10, you could offset most of that with better codecs. (Credit: commenter petrilli on Schneier's recent post about this.)
guard-of-terra 13y ago
Any B2C calls are reflected (well, half-reflected actually) on this sheet and B2B don't talk so much anyway, I guess.

maresca 13y ago

The point is that even quadrupled, the number is so low, it's a drop in the bucket for the government.

tommi 13y ago

Napkin math sure, but not at its worst. It's just an very rough estimation with explanations for numbers given and not claiming to be exact. I think the most valuable lesson here is that it's likely to be in the ballpark of millions of dollars and that in theory its possible to do it for $27.2 million.

arscan 13y ago

I think the point here is that its not some ridiculously huge number, like $100 billion. It is feasible for the NSA to actually store a substantial percentage of US phone calls if they wanted.

rm999 13y ago

It's safe to assume these kinds of estimates are "order of magnitude" estimates, meaning they could be off by a factor of 2 or more. The goal isn't to build your own system or to bill the government, it's to put the numbers in perspective. Case in point: if you had asked me if phone recording cost closer to 10 billion dollars or 100 million dollars I would have guessed 10 billion; now I know better. Being able to quickly do order of magnitude estimates (especially in your head) is a very important skill IMO. I've heard McKinsey makes it part of their interview process.

larrys 13y ago
Also need to tack on the cost of developing the system, oversight, bidding etc. Still I would have guess much higher as well.

ctdonath 13y ago

Napkin math still gets a ballpark notion. Let's compute an outer limit: record everyone, all the time, CD quality. 44100 samples per second * 2 bytes per sample * 2 channels * 60 seconds per minute * 60 minutes per hour * 24 hours per day * 365 days per year * 313900000 people * $100 per terabyte / 1 terabyte = $175 billion per year. That's an absolute outer limit for cost. That's less than 5% of federal budget, and we haven't started on the obvious ways to cut costs by several orders of magnitude. Reduce it to 4410 samples/sec, 1 byte per sample, 1 channel, 1/10th the time and we're already under $0.5B/yr, without even addressing audio compression (much less voice-to-text).

bradleyland 13y ago
So what inference are we drawing? I honestly wasn't aware that the capability to store all voice traffic was in question. That might be owed to my background in telecom though. Telephone codecs are extremely low bitrate (relative to something like music, much less video) because you have very clear design constraints, and those constraints are forgiving. You just need to be able to understand the caller's voice, not accurately reproduce a live performance of Beethoven's 5th. I agree that the estimates to store the data are on the very, very high side, but I'm not sure that number is even significant in the whole scope of the challenge. The challenges in obtaining every phone call made in the US aren't storage related; they're almost entirely collection and aggregation related. These ballpark numbers don't even attempt to factor in that portion of the cost. IMO, the collection, transport, and aggregation systems are easily 8/10ths of the problem, not storage. Storage is an extremely scalable solution. If the government were going to do something like this, they'd likely tap in to the phone networks at the same places everyone else does. There are major telecom aggregation points - called "tandems" [1] - around the country. If you want to set up your own phone carrier with an actual physical network of your own, this is where you plug in. The government would have to do the same. They can't simply tap in at a single aggregation point, because not all phone calls pass through the same points. From there, you face the choice of storing the data regionally in several data centers, or attempting to aggregate it all back to a central data center. IMO, a disaggregated approach makes a lot more sense. You could reasonably expect to transport all CDR [2] data back to a central location, but you wouldn't want to send all media (audio data) back to one place. It would be simple enough to only fetch what you need based on a query against call data. You'd want all the the CDR data in one place so you could perform your "big data" analysis on it efficiently, then cherry pick media to pull in for analysis. I'm certain I haven't even scratched the surface here. This only gets the government long distance calls. It doesn't touch local, or even intraLATA calls. Maybe the government isn't interested in those calls though. Maybe they're only interested in international calls, which makes the problem simpler, not harder. Others in the thread have mentioned that this should be treated as an "order of magnitude" estimation. I don't think we can draw any inference from this estimation at all, because it represents such a small portion of the problem domain. We also have no idea what the scope of the challenge is. The entire exercise is pointless when you think about what we're asking. "Can the government actually implement a solution to record every phone call in the US?" I think that's a resounding yes. The solution would look a lot like setting up a tier 2 network provider [3] with extra investment in a storage back end. That's entirely within the realm of possibility given the size of the US Dept of Defense budget. [1] http://en.wikipedia.org/wiki/Class_4_telephone_switch http://en.wikipedia.org/wiki/Class_4_telephone_switch [2] http://en.wikipedia.org/wiki/Call_detail_record http://en.wikipedia.org/wiki/Call_detail_record [3] http://en.wikipedia.org/wiki/Tier_2_network http://en.wikipedia.org/wiki/Tier_2_network

dagw 13y ago

Sure, but the point doesn't change. All the calls made in the US can be stored on a budget that isn't out of reach for an organization like the NSA. Napkin math is all about getting you within the right order of magnitude and this manages that just fine.

skue 13y ago

OTOH, the estimated bytes per call truly is conservative. The 8KB/s estimate is based on G.711, and using G.726 or G.729 would yield a 2-8x reduction in storage. So this is definitely within an order of magnitude: even if the minutes of talk time are off by a factor of 10, you could offset most of that with better codecs. (Credit: commenter petrilli on Schneier's recent post about this.)

guard-of-terra 13y ago

Any B2C calls are reflected (well, half-reflected actually) on this sheet and B2B don't talk so much anyway, I guess.