5 ms·

"in several recent years NSA has been the largest single buyer of spinning disks in the world." The rest of the world moved to SSDs in ~2010. Google's been st

"in several recent years NSA has been the largest single buyer of spinning disks in the world." The rest of the world moved to SSDs in ~2010. Google's been storing the whole web on flash since then. (I guess if you're archiving for slow-speed retrieval disks are still useful, but if that's your use-case, why wouldn't you use tape?)

_jal 6y ago
> why wouldn't you use tape Because there are many orders of latency magnitude between spinning rust and tape. We still use a lot of spinning disk for precisely this reason - you get roughly 5x the online capacity for the same cost and automation does not whine about wanting the new shiny.
- nostrademons 6y ago
  That was why I mentioned SSDs - there are orders of latency magnitude between SSDs and spinning disk. I guess it fits the NSA's use-case though, where they have ~thousands of analysts running complex queries but no external users, rather than ~billions of external users running well-defined queries.
magila 6y ago
I left the storage industry about 5 years ago, but at least as of then the big cloud providers were still buying lots of spinning rust and showed no signs of stopping. These companies store a lot of long tail data which needs to be available but doesn't need super low latency access. It's still much more economical to store such data on disks than in flash.
jasonwatkinspdx 6y ago
I wasn't really trying to get into some sort of debate about storage technologies, just illustrating that the NSA does have a whole lot of data. Spinning disk is cost competitive with tape robots these days, while offering much better access performance. There's a few different vendors that supply this sort of setup, or you can build your own backblaze style. The core idea is you build storage servers that can hold more disks than can be active at any given moment. Powering up a drive and doing a seek is a lot faster than a tape robot.
specktr 6y ago
I’ve worked on infrastructure (design) engineering teams at a very large company that still does their own datacenters. I can confirm that spinners are still very much in use and are not used for archival purposes - that’s what tapes are designed and used for.

_jal 6y ago

> why wouldn't you use tape Because there are many orders of latency magnitude between spinning rust and tape. We still use a lot of spinning disk for precisely this reason - you get roughly 5x the online capacity for the same cost and automation does not whine about wanting the new shiny.

nostrademons 6y ago
That was why I mentioned SSDs - there are orders of latency magnitude between SSDs and spinning disk. I guess it fits the NSA's use-case though, where they have ~thousands of analysts running complex queries but no external users, rather than ~billions of external users running well-defined queries.

magila 6y ago

I left the storage industry about 5 years ago, but at least as of then the big cloud providers were still buying lots of spinning rust and showed no signs of stopping. These companies store a lot of long tail data which needs to be available but doesn't need super low latency access. It's still much more economical to store such data on disks than in flash.

jasonwatkinspdx 6y ago

I wasn't really trying to get into some sort of debate about storage technologies, just illustrating that the NSA does have a whole lot of data. Spinning disk is cost competitive with tape robots these days, while offering much better access performance. There's a few different vendors that supply this sort of setup, or you can build your own backblaze style. The core idea is you build storage servers that can hold more disks than can be active at any given moment. Powering up a drive and doing a seek is a lot faster than a tape robot.

specktr 6y ago

I’ve worked on infrastructure (design) engineering teams at a very large company that still does their own datacenters. I can confirm that spinners are still very much in use and are not used for archival purposes - that’s what tapes are designed and used for.