7 ms·

He supposedly had 50TB+ worth of data sitting at his place. This almost seems inaccurate as that's like 25 x 2TB drives. I know this is said to have happened ov

He supposedly had 50TB+ worth of data sitting at his place. This almost seems inaccurate as that's like 25 x 2TB drives. I know this is said to have happened over 20 years, but hard drive capacity has only increased within the last few years to where this is manageable. He must have had a huge storage array. Logistically, how do you even walk out with that? Cannot transfer it over the network as that would take years and surely not go unnoticed. This included 75% of exploits used by the NSA. Just seems so crazy that this could go unnoticed for so long. You think they would be watching everything like a hawk. http://boingboing.net/2016/10/20/nsa-contractor-harold-thomas-m.html http://boingboing.net/2016/10/20/nsa-contractor-harold-thoma...

austincheney 10y ago
You couldn't transfer it over the network anyways, because the networks are physically separated from the normal internet (NIPR). You cannot use massive storage devices either (like hard disks, sd cards, flash drives). He would have had to hand carry stuff out, probably on optical media.
- WestCoastJustin 10y ago
  Ah, great points. Thanks.
- digler999 10y ago
  > probably on optical media. I cringe thinking NSA would have DVD-burners connected to any machine on their campus. Sure in their labs they probably have every device ever made -- strictly labeled and used only for hacking. But it would be asinine to have their "corporate" machines equipped with them. I bet he either used a microSD card or just put HDDs in his pocket.
  - austincheney 10y ago
    You need optical media to transfer content from one network to another without setting off alarms. The network admin knows if you plug in mass storage devices within about 2 seconds. Mass storage devices require a mounting process that communicates with both the bios and the OS. The OS will then send an alert to a known point on the network, and if configured directly will respond to the switch at the local network which will drop the subnet. If you are dumb enough to pop a USB device in everybody in your office will know in seconds as they lose their connection. Then your boss will get scolded and you will get stuck with remedial training. It is pretty damn embarrassing like wearing a dunce cap. They may not always know who the offending person is, but they will have the IP of the offending terminal and a hardware signature of the offending device.
    - openmosix 10y ago
      Yup, many organizations have this policy enforced. It'd be crazy to think a place like NSA would let you connect your USB stick to any of their terminals.
    - balabaster 10y ago
      What if he had a legit reason for connecting mass storage, but just abused the privilege while connected? Network admins know he's connecting it, know what he says he's doing, has approved it and thus overrides and ignores any warnings... Equally, what's to stop them being in the know of exactly what he's doing and abusing their privilege to allow him to do it? You assume he's acting alone... what if he wasn't? The best lies contain elements of truth... or are almost entirely true. Find a legit reason for bypassing security and while you're bypassed, abuse it without getting caught.
      - austincheney 10y ago
        > Equally, what's to stop them being in the know of exactly what he's doing and abusing their privilege to allow him to do it? You assume he's acting alone... what if he wasn't? Because violations and exceptions to policy are logged and those logs are periodically audited. There is a process for achieving access with an exception to policy for various approved reasons. Deleting entries from access control logs requires an accomplice that doesn't mind jail time. At some point this starts to sound like a crazy conspiracy theory. I find the most simple answers are generally the most probable. Let's not forget this went on for nearly 20 years and security staff gets rotated.
  - makomk 10y ago
    They have to transfer data from the high side to the low side somehow, and supposedly carefully-scrutinized DVD burners is one way of doing this.
noonespecial 10y ago
Until I learn more I'm going to take the 50TB accusation like the (as much as) 300 years jail time they're likely threatening him with to get the plea deal. Sound like there might be some funny counting going on.
- jlgaddis 10y ago
  > 300 years $ echo 20*10 | bc 200 Funny counting indeed. <g>
- digler999 10y ago
  it wouldn't be surprising that a guy who spent his life in infosec/IT might have a 50tb NAS at home. My guess is he was addicted to "stealing", the same way some people shoplift for the thrill. It can feel empowering to steal because it gives the thief control. So subconsciously he's thinking "huh, NSA thinks they're so tough and powerful, but look how I can break their rules right under their nose and they can't even figure it out". Or perhaps he exfiltrated people's phone calls/emails. Thats another power theory: "I'm as big and mighty as the NSA because I can tap communication lines and invade privacy just like they can". Like others in this thread, I dont think he was a spy. A spy wouldn't hoard shit in his basement that could get him life in prison. He would have stolen it, sold it, and GTFO.
  - noonespecial 10y ago
    Oh he might have, but I'm guessing they just added up all of the space from every hard drive they found searching his house to get that number. Well, he's a thief, they all could have teh secritz on them.
ohyes 10y ago
50 terabytes wouldn't take years to transfer over the Internet, maybe a month or so.
- WestCoastJustin 10y ago
  Yeah, I was kind of thinking about going unnoticed though. I was involved in a project a few years ago where we transferred 500TB to a new data centre and it took about 3 months. You're going to saturate pipes and have things pinned using multiple streams. Definitely not using a home connection. So, you'd want to fly under the radar, and really slow things down. That's where my head was at when making that comment. As one of the replies mentions though, you cannot even transfer over the network, as they are physically separated.
  - drvdevd 10y ago
    I think you're right.. The numbers don't seem to add up. They seem like inflated "media" numbers. The media/general public just knows 1TB is "big" (because most people just need gigabytes) and 50 times that is Serious Business.
jasonkostempski 10y ago
They probably uncompressed all data and stretched every jpg to poster size and converted it to bmp to make the number sound bigger.
madiathomas 10y ago
Intelligence agencies are always years ahead of us in terms of technology. They probably had access to 2TB hard drives 5-10 years before us.
- throwaway2048 10y ago
  its extremely doubtful they are that far ahead when commerical offerings have been competing for any edge for the last 30+ years.
Retric 10y ago
DVD's hold up to 8.5GB. So under 6,000 DVD's which sounds like a lot, but at under 1$ each so it's only like 5k. A spindle can hold 100 and you get more than 4 of them per cubic foot. So call it 15 cubic feet of DVD's that's basically just a single plastic tub. Granted, more than that with CD's and less than that with Blue Ray disks. But, if he took burnable DVD's home regularly. Anyway, it would be interesting if he had a fetish for this stuff, wanted to sell it, or was just being really stupid. PS: Only idea I have for a dumb, but plausible explication is if he was backing up his classified work laptop every week or something.
- rising-sky 10y ago
  It's doubtful he wanted to sell it, if the information was of any value, rhetorical, considering the NSA is up in arms about it, then he wouldn't have had trouble finding a willing and able buyer in all of those 20 years. *edited for grammar
  - jdironman 10y ago
    If he actually did it, I believe it may have been for present documentation and future accountability. Nothing at the time, but over time it became a habit. Which might have explained how quickly he admitted once confronted with the original evidence of 6 documents. But anyways, see my other top level comment for what I really want to know.
FLUX-YOU 10y ago
His rate of moving data could have increased as storage density increased. It's probably not 50TB / 20 years = rate
anigbrowl 10y ago
He supposedly had 50TB+ worth of data sitting at his place. This almost seems inaccurate as that's like 25 x 2TB drives. My wife works in electronic discovery, processing huge caches of corporate documents for complex litigation. 50tb is a lot of data but it's not that much in organizational terms. From what I remember when her firm was processing material for the SEC-Goldman Sachs litigation over the financial crisis (a case which involved several years of discovery work), they were dealing with a roughly similar quality, 30 or 60tb of data. I've got an unopened 5tb drive sitting right here on my desk. No offense, but I feel you're arguing against the existence of large trucks because you've never had any reason to drive one. You think they would be watching everything like a hawk. You would, but in my experience managers pay about as much attention to IT people as they do to the janitors. That goes double for contractors, because the buyers tend to assume security screening and so on was built into the terms of the contract. At one point I was the only IT guy for a small private bank in London; I was contracted to work for Digital Equipment Corp, who in turn had the corporate contract with this bank due the contractual arrangements between DEC and the much larger corporate parent of the private bank. I got the job because their existing guy fired or quit in a hurry or something and they were desperate. Technically it was easy, I just ran backups, did a few service calls around the office each day if anyone had a problem, and most of the actual work consisted of passing on the bank's procurement requests to the DEC people. Eventually I lost the gig because the bank got sick of dealing with DEC and terminated the contract but I got glowing personal references which was nice. I remember being sort of amazed at the degree of freedom I had. As you may know, small private banks cater exclusively to the ultra-rich, and I was the only IT person on site and had total access to anything. If I were more financially oriented I could have enriched myself in any number of ways.
- WestCoastJustin 10y ago
  > No offense, but I feel you're arguing against the existence of large trucks because you've never had any reason to drive one. I'm not saying that companies don't have 50TB of documents, or that you cannot transfer 50TB, either via the network or physically. I am saying that for a place which prides itself as the "National Security Agency", they should have extremely tight security about transferring 50TB of their crown jewels, either via the network or physically. So, I think we are talking about different things. Goldman Sachs and DEC are in totally different ballparks, or should be, compared to the NSA. I don't really care either way. I just think logistically this is a crazy amount of data for someone to walk off with at a place that should be the model of a secure site.
jlgaddis 10y ago
From The Verge article [0]: > It was allegedly all kept on computers and drives at Martin’s Glen Burnie, Maryland home. [0]: http://www.theverge.com/2017/2/8/14555238/nsa-leak-indictment-harold-thomas-martin-20-charges http://www.theverge.com/2017/2/8/14555238/nsa-leak-indictmen...
metaphor 10y ago
What makes you think the data was all stored on standard hard drives? How about: CDs, DVDs, Blu-Rays, magneto-optical disks, Zip disks, USB thumb drives, high-density archive media that I'm not even familiar with...oh, and hard drives. What does huge storage array even mean in today's context? I recently built a 20TB server at a price point less than a new 13" MacBook Pro for personal use that occupies a cross section the size of a notebook sitting at the edge my desk...all from commercial-grade hardware that's been on the market for at least 2 years. I would think that the home setup of any serious professional engineer who has been in industry for the better part of my adult life would have some sweet kit up his sleeves. Imagine this: authorities confiscate anything that looks like storage media, storage capacity rough order of magnitude is calculated, figures reported to media, an assumption is made that it's all highly classified, spin some click bait article. We're talking about a civilian contractor from private industry who supposedly played an "advisor" role, not some intern whose shoulder was constantly being looked over.
- cyberpunk 10y ago
  20TB less than the size of a notebook? I'm not calling outright fibs here but we're talking 10 2TB sata's here or what? I'd take maybe a 1U with SFF's, but then again I've not seen how big your laptop is ;) edit: yep, ok, I missed the words 'server' and 'cross section' there and thought you were talking about some magic home setup...... apologies and: doh!
  - ewams 10y ago
    20TB of data can fit on 3 hard drives now...
    - cyberpunk 10y ago
      I've clearly not been keeping up.. Assuming a stripe then we're talking 7-8TB drives? How much do those cost? I guess that's a kind of expensive setup for what's being suggested was his home setup though? 20TB in a mirror is cheap as in the DC, at home I've not pushed above about 5 (don't have that much data and thus won't spank so much money on capacity)... Are you running these things?
xjwm 10y ago
>You think they would be watching everything like a hawk. They probably do...but...as they say, "locks keep honest men honest." At some point, you have to have a level of trust that exists. A long term subersive employee will beat your locks eventually.
harry8 10y ago
Let's just say an average txt doc is 4k, and an average uncompressed bitmap of that same document is 40M. Documents with contents that would cost 50 terrabytes to scan and store as uncompressed tiff or whatever cost 5G as ascii. This is by way of illustration only. Just think "arrested with drugs having a street value of $10M" and ask yourself if that same kind of inflation is likely to be going on here in the prosecutions PR campaign - which is what this story is.[1] [1] Note that this could be a fair, reasonable, unbiased and accurate story, but it would be a heroic assumption not to consider the alternatives.
tw04 10y ago
I doubt all 50TB was collected in the first year. The 2TB hard drive was released in 2009... 8 years ago. I would be willing to bet his stash increased at a similar rate to drive sizes, if for no other reason than the fact that NSA data collections increased at what is likely a similar rate over that time. A year ago 6TB drives were already widely available, so now you're down to 10 drives. In other words, a medium sized home NAS for most nerds.
astronautjones 10y ago
They certainly have a history of exaggerating numbers
ynniv 10y ago
SD cards today come in 512GB, so with two of them you can casually move 50TB+ in three months. Everyone underestimates sneakernet.

austincheney 10y ago

You couldn't transfer it over the network anyways, because the networks are physically separated from the normal internet (NIPR). You cannot use massive storage devices either (like hard disks, sd cards, flash drives). He would have had to hand carry stuff out, probably on optical media.

WestCoastJustin 10y ago
Ah, great points. Thanks.
digler999 10y ago
> probably on optical media. I cringe thinking NSA would have DVD-burners connected to any machine on their campus. Sure in their labs they probably have every device ever made -- strictly labeled and used only for hacking. But it would be asinine to have their "corporate" machines equipped with them. I bet he either used a microSD card or just put HDDs in his pocket.
- austincheney 10y ago
  You need optical media to transfer content from one network to another without setting off alarms. The network admin knows if you plug in mass storage devices within about 2 seconds. Mass storage devices require a mounting process that communicates with both the bios and the OS. The OS will then send an alert to a known point on the network, and if configured directly will respond to the switch at the local network which will drop the subnet. If you are dumb enough to pop a USB device in everybody in your office will know in seconds as they lose their connection. Then your boss will get scolded and you will get stuck with remedial training. It is pretty damn embarrassing like wearing a dunce cap. They may not always know who the offending person is, but they will have the IP of the offending terminal and a hardware signature of the offending device.
  - openmosix 10y ago
    Yup, many organizations have this policy enforced. It'd be crazy to think a place like NSA would let you connect your USB stick to any of their terminals.
  - balabaster 10y ago
    What if he had a legit reason for connecting mass storage, but just abused the privilege while connected? Network admins know he's connecting it, know what he says he's doing, has approved it and thus overrides and ignores any warnings... Equally, what's to stop them being in the know of exactly what he's doing and abusing their privilege to allow him to do it? You assume he's acting alone... what if he wasn't? The best lies contain elements of truth... or are almost entirely true. Find a legit reason for bypassing security and while you're bypassed, abuse it without getting caught.
    - austincheney 10y ago
      > Equally, what's to stop them being in the know of exactly what he's doing and abusing their privilege to allow him to do it? You assume he's acting alone... what if he wasn't? Because violations and exceptions to policy are logged and those logs are periodically audited. There is a process for achieving access with an exception to policy for various approved reasons. Deleting entries from access control logs requires an accomplice that doesn't mind jail time. At some point this starts to sound like a crazy conspiracy theory. I find the most simple answers are generally the most probable. Let's not forget this went on for nearly 20 years and security staff gets rotated.
- makomk 10y ago
  They have to transfer data from the high side to the low side somehow, and supposedly carefully-scrutinized DVD burners is one way of doing this.

noonespecial 10y ago

Until I learn more I'm going to take the 50TB accusation like the (as much as) 300 years jail time they're likely threatening him with to get the plea deal. Sound like there might be some funny counting going on.

jlgaddis 10y ago
> 300 years $ echo 20*10 | bc 200 Funny counting indeed. <g>
digler999 10y ago
it wouldn't be surprising that a guy who spent his life in infosec/IT might have a 50tb NAS at home. My guess is he was addicted to "stealing", the same way some people shoplift for the thrill. It can feel empowering to steal because it gives the thief control. So subconsciously he's thinking "huh, NSA thinks they're so tough and powerful, but look how I can break their rules right under their nose and they can't even figure it out". Or perhaps he exfiltrated people's phone calls/emails. Thats another power theory: "I'm as big and mighty as the NSA because I can tap communication lines and invade privacy just like they can". Like others in this thread, I dont think he was a spy. A spy wouldn't hoard shit in his basement that could get him life in prison. He would have stolen it, sold it, and GTFO.
- noonespecial 10y ago
  Oh he might have, but I'm guessing they just added up all of the space from every hard drive they found searching his house to get that number. Well, he's a thief, they all could have teh secritz on them.

ohyes 10y ago

50 terabytes wouldn't take years to transfer over the Internet, maybe a month or so.

WestCoastJustin 10y ago
Yeah, I was kind of thinking about going unnoticed though. I was involved in a project a few years ago where we transferred 500TB to a new data centre and it took about 3 months. You're going to saturate pipes and have things pinned using multiple streams. Definitely not using a home connection. So, you'd want to fly under the radar, and really slow things down. That's where my head was at when making that comment. As one of the replies mentions though, you cannot even transfer over the network, as they are physically separated.
- drvdevd 10y ago
  I think you're right.. The numbers don't seem to add up. They seem like inflated "media" numbers. The media/general public just knows 1TB is "big" (because most people just need gigabytes) and 50 times that is Serious Business.

jasonkostempski 10y ago

They probably uncompressed all data and stretched every jpg to poster size and converted it to bmp to make the number sound bigger.

madiathomas 10y ago

Intelligence agencies are always years ahead of us in terms of technology. They probably had access to 2TB hard drives 5-10 years before us.

throwaway2048 10y ago
its extremely doubtful they are that far ahead when commerical offerings have been competing for any edge for the last 30+ years.

Retric 10y ago

DVD's hold up to 8.5GB. So under 6,000 DVD's which sounds like a lot, but at under 1$ each so it's only like 5k. A spindle can hold 100 and you get more than 4 of them per cubic foot. So call it 15 cubic feet of DVD's that's basically just a single plastic tub. Granted, more than that with CD's and less than that with Blue Ray disks. But, if he took burnable DVD's home regularly. Anyway, it would be interesting if he had a fetish for this stuff, wanted to sell it, or was just being really stupid. PS: Only idea I have for a dumb, but plausible explication is if he was backing up his classified work laptop every week or something.

rising-sky 10y ago
It's doubtful he wanted to sell it, if the information was of any value, rhetorical, considering the NSA is up in arms about it, then he wouldn't have had trouble finding a willing and able buyer in all of those 20 years. *edited for grammar
- jdironman 10y ago
  If he actually did it, I believe it may have been for present documentation and future accountability. Nothing at the time, but over time it became a habit. Which might have explained how quickly he admitted once confronted with the original evidence of 6 documents. But anyways, see my other top level comment for what I really want to know.

FLUX-YOU 10y ago

His rate of moving data could have increased as storage density increased. It's probably not 50TB / 20 years = rate

anigbrowl 10y ago

He supposedly had 50TB+ worth of data sitting at his place. This almost seems inaccurate as that's like 25 x 2TB drives. My wife works in electronic discovery, processing huge caches of corporate documents for complex litigation. 50tb is a lot of data but it's not that much in organizational terms. From what I remember when her firm was processing material for the SEC-Goldman Sachs litigation over the financial crisis (a case which involved several years of discovery work), they were dealing with a roughly similar quality, 30 or 60tb of data. I've got an unopened 5tb drive sitting right here on my desk. No offense, but I feel you're arguing against the existence of large trucks because you've never had any reason to drive one. You think they would be watching everything like a hawk. You would, but in my experience managers pay about as much attention to IT people as they do to the janitors. That goes double for contractors, because the buyers tend to assume security screening and so on was built into the terms of the contract. At one point I was the only IT guy for a small private bank in London; I was contracted to work for Digital Equipment Corp, who in turn had the corporate contract with this bank due the contractual arrangements between DEC and the much larger corporate parent of the private bank. I got the job because their existing guy fired or quit in a hurry or something and they were desperate. Technically it was easy, I just ran backups, did a few service calls around the office each day if anyone had a problem, and most of the actual work consisted of passing on the bank's procurement requests to the DEC people. Eventually I lost the gig because the bank got sick of dealing with DEC and terminated the contract but I got glowing personal references which was nice. I remember being sort of amazed at the degree of freedom I had. As you may know, small private banks cater exclusively to the ultra-rich, and I was the only IT person on site and had total access to anything. If I were more financially oriented I could have enriched myself in any number of ways.

WestCoastJustin 10y ago
> No offense, but I feel you're arguing against the existence of large trucks because you've never had any reason to drive one. I'm not saying that companies don't have 50TB of documents, or that you cannot transfer 50TB, either via the network or physically. I am saying that for a place which prides itself as the "National Security Agency", they should have extremely tight security about transferring 50TB of their crown jewels, either via the network or physically. So, I think we are talking about different things. Goldman Sachs and DEC are in totally different ballparks, or should be, compared to the NSA. I don't really care either way. I just think logistically this is a crazy amount of data for someone to walk off with at a place that should be the model of a secure site.

jlgaddis 10y ago

From The Verge article [0]: > It was allegedly all kept on computers and drives at Martin’s Glen Burnie, Maryland home. [0]: http://www.theverge.com/2017/2/8/14555238/nsa-leak-indictment-harold-thomas-martin-20-charges http://www.theverge.com/2017/2/8/14555238/nsa-leak-indictmen...

metaphor 10y ago

What makes you think the data was all stored on standard hard drives? How about: CDs, DVDs, Blu-Rays, magneto-optical disks, Zip disks, USB thumb drives, high-density archive media that I'm not even familiar with...oh, and hard drives. What does huge storage array even mean in today's context? I recently built a 20TB server at a price point less than a new 13" MacBook Pro for personal use that occupies a cross section the size of a notebook sitting at the edge my desk...all from commercial-grade hardware that's been on the market for at least 2 years. I would think that the home setup of any serious professional engineer who has been in industry for the better part of my adult life would have some sweet kit up his sleeves. Imagine this: authorities confiscate anything that looks like storage media, storage capacity rough order of magnitude is calculated, figures reported to media, an assumption is made that it's all highly classified, spin some click bait article. We're talking about a civilian contractor from private industry who supposedly played an "advisor" role, not some intern whose shoulder was constantly being looked over.

cyberpunk 10y ago
20TB less than the size of a notebook? I'm not calling outright fibs here but we're talking 10 2TB sata's here or what? I'd take maybe a 1U with SFF's, but then again I've not seen how big your laptop is ;) edit: yep, ok, I missed the words 'server' and 'cross section' there and thought you were talking about some magic home setup...... apologies and: doh!
- ewams 10y ago
  20TB of data can fit on 3 hard drives now...
  - cyberpunk 10y ago
    I've clearly not been keeping up.. Assuming a stripe then we're talking 7-8TB drives? How much do those cost? I guess that's a kind of expensive setup for what's being suggested was his home setup though? 20TB in a mirror is cheap as in the DC, at home I've not pushed above about 5 (don't have that much data and thus won't spank so much money on capacity)... Are you running these things?

xjwm 10y ago

>You think they would be watching everything like a hawk. They probably do...but...as they say, "locks keep honest men honest." At some point, you have to have a level of trust that exists. A long term subersive employee will beat your locks eventually.

harry8 10y ago

Let's just say an average txt doc is 4k, and an average uncompressed bitmap of that same document is 40M. Documents with contents that would cost 50 terrabytes to scan and store as uncompressed tiff or whatever cost 5G as ascii. This is by way of illustration only. Just think "arrested with drugs having a street value of $10M" and ask yourself if that same kind of inflation is likely to be going on here in the prosecutions PR campaign - which is what this story is.[1] [1] Note that this could be a fair, reasonable, unbiased and accurate story, but it would be a heroic assumption not to consider the alternatives.

tw04 10y ago

I doubt all 50TB was collected in the first year. The 2TB hard drive was released in 2009... 8 years ago. I would be willing to bet his stash increased at a similar rate to drive sizes, if for no other reason than the fact that NSA data collections increased at what is likely a similar rate over that time. A year ago 6TB drives were already widely available, so now you're down to 10 drives. In other words, a medium sized home NAS for most nerds.

astronautjones 10y ago

They certainly have a history of exaggerating numbers

ynniv 10y ago

SD cards today come in 512GB, so with two of them you can casually move 50TB+ in three months. Everyone underestimates sneakernet.