5 ms·

How much would you pay to store such amount data? 8TB hard drive is about 200 dollars nowadays, so this amount of data is about 200m dollars worthy of retail h

How much would you pay to store such amount data? 8TB hard drive is about 200 dollars nowadays, so this amount of data is about 200m dollars worthy of retail hard drives. Ofc the calculation is super inaccurate, which doesn't take redundancies into consideration, and discounted prices for someone like Google to purchase hardware, plus the discount for Apple as a big customer. But had the scale be comparable, in which case, Apple is paying Google billions per year to handle the data storage for itself, doesn't sound quite news worthy IMO, pretty price efficient even. Edit: Previously stated 1.6B -> 200m

gruez 5y ago
>8TB hard drive is about 200 dollars nowadays, so this amount of data is about 1.6B dollars worthy of retail hard drives. External hard drives can be obtained on sale for $15-16/TB. They can then be shucked to get internal drives. I suspect bulk buyers can get pricing equal or cheaper than this, rather than the retail of $25/TB
- hinkley 5y ago
  They are probably paying wholesale prices for that quantity. In many industries that’s half the retail price.
- wincy 5y ago
  Why are external drives so much cheaper anyway? Edit: I googled it an apparently it’s an economy of scale thing. People are more likely to buy external hard drives, much more mass market thing, Costco and Walmart sell externals, therefore downward pressure on prices compared to internal drives.
  - ip26 5y ago
    External drives are also spec'd for lower performance and longevity.
    - gruez 5y ago
      Do external drives even have performance/longevity specs? The specs I see on them are pretty basic (eg. capacity, interface, operating temperature), and considering that they're marketed to consumers that's not surprising. As for actual performance/longevity, my understanding is that they're just whatever drives the manufacturer has available. For low capacities they're going to be consumer drives and for higher capacities they'll usually be NAS/datacenter drives.
      - ip26 5y ago
        whatever drives the manufacturer has available Exactly, that’s a spec :)
- spookthesunset 5y ago
  At the scale these data centers operate at, I would imagine that google / FB / AWS gets these storage devices custom built to whatever spec and tolerances they need.
DenseComet 5y ago
The article claims that Apple is on track to spend around $300 million on GCP storage services this year. With redundancies, server costs, and DC costs, the upfront price could easily touch 3 to 6 billion, or 10 to 20 years of GCP storage costs.
- mlyle 5y ago
  > With redundancies, server costs, and DC costs, the upfront price could easily touch 3 to 6 billion, or 10 to 20 years of GCP storage costs. Surely it didn't cost GOOG $6B upfront worth of infrastructure to get $300M of annual revenue from Apple. Yes, there's economies of scale, but I think you overstate the case.
  - jfrunyon 5y ago
    Google already has a lot of the infrastructure and labor needed.
selcuka 5y ago
Shouldn't that be $200M?
panarky 5y ago
Plus bandwidth
- jhgb 5y ago
  And locations, surely? To minimize latencies.
discodave 5y ago
Edit: I did this calculation for the Dropbox migration out of S3 in this blog post: http://blog.drgriffin.com.au/posts/2020-06-21-the-three-fs-of-cloud-pricing.html http://blog.drgriffin.com.au/posts/2020-06-21-the-three-fs-o... Short answer: The cost of just the hard drives to replace an $89MM S3 storage bill was about $45 million dollars. That's not including bandwidth, racks, datacenter space, administration etc etc.
KaiserPro 5y ago
The secret sauce here is the opportunity cost of running one's own storage at that scale. At that scale its less about sourcing the hardrives (yes its a problem, but now youre a big customer so you can ask for specialised things, and a 50% discount) The big problem is the hosting, finding and building a number of datacentres local to where your customers are. Powering them, and connecting in a reliable way. But even more, you need to make a storage interface that doesn't suck. I am currently working with a homegrown S3 "like" interface. It is a massive pain in the arse as it doesn't scale[1], has an entirely new set of words to describe standard things (trying to overwrite a file? it doesn't tell you that, it just says "predicate failed") [1] large files transfer very fast, faster than s3. However all the tools are written single threaded. Add to this that each operation takes at least .7-1.5 seconds, shit gets slow very quickly. There is little to no documentation, the API is odd. in short, much as it annoys me to say this: for general purpose storage, buy over build.
spookthesunset 5y ago
Is that stuff even stored on spinning rust these days? I mean solid state might be more expensive upfront but I imagine at scale the energy consumption required to power and cool millions of mechanical devices becomes a big issue. Not to mention the random access performance of spiny disks ain’t too great either.
- discodave 5y ago
  Yes, spinning rust is still used for low cost storage. You can look up Dropbox Magic Pocket and I think some stuff that AWS has made public about S3.

gruez 5y ago

>8TB hard drive is about 200 dollars nowadays, so this amount of data is about 1.6B dollars worthy of retail hard drives. External hard drives can be obtained on sale for $15-16/TB. They can then be shucked to get internal drives. I suspect bulk buyers can get pricing equal or cheaper than this, rather than the retail of $25/TB

hinkley 5y ago
They are probably paying wholesale prices for that quantity. In many industries that’s half the retail price.
wincy 5y ago
Why are external drives so much cheaper anyway? Edit: I googled it an apparently it’s an economy of scale thing. People are more likely to buy external hard drives, much more mass market thing, Costco and Walmart sell externals, therefore downward pressure on prices compared to internal drives.
- ip26 5y ago
  External drives are also spec'd for lower performance and longevity.
  - gruez 5y ago
    Do external drives even have performance/longevity specs? The specs I see on them are pretty basic (eg. capacity, interface, operating temperature), and considering that they're marketed to consumers that's not surprising. As for actual performance/longevity, my understanding is that they're just whatever drives the manufacturer has available. For low capacities they're going to be consumer drives and for higher capacities they'll usually be NAS/datacenter drives.
    - ip26 5y ago
      whatever drives the manufacturer has available Exactly, that’s a spec :)
spookthesunset 5y ago
At the scale these data centers operate at, I would imagine that google / FB / AWS gets these storage devices custom built to whatever spec and tolerances they need.

DenseComet 5y ago

The article claims that Apple is on track to spend around $300 million on GCP storage services this year. With redundancies, server costs, and DC costs, the upfront price could easily touch 3 to 6 billion, or 10 to 20 years of GCP storage costs.

mlyle 5y ago
> With redundancies, server costs, and DC costs, the upfront price could easily touch 3 to 6 billion, or 10 to 20 years of GCP storage costs. Surely it didn't cost GOOG $6B upfront worth of infrastructure to get $300M of annual revenue from Apple. Yes, there's economies of scale, but I think you overstate the case.
- jfrunyon 5y ago
  Google already has a lot of the infrastructure and labor needed.

selcuka 5y ago

Shouldn't that be $200M?

panarky 5y ago

Plus bandwidth

jhgb 5y ago
And locations, surely? To minimize latencies.

discodave 5y ago

Edit: I did this calculation for the Dropbox migration out of S3 in this blog post: http://blog.drgriffin.com.au/posts/2020-06-21-the-three-fs-of-cloud-pricing.html http://blog.drgriffin.com.au/posts/2020-06-21-the-three-fs-o... Short answer: The cost of just the hard drives to replace an $89MM S3 storage bill was about $45 million dollars. That's not including bandwidth, racks, datacenter space, administration etc etc.

KaiserPro 5y ago

The secret sauce here is the opportunity cost of running one's own storage at that scale. At that scale its less about sourcing the hardrives (yes its a problem, but now youre a big customer so you can ask for specialised things, and a 50% discount) The big problem is the hosting, finding and building a number of datacentres local to where your customers are. Powering them, and connecting in a reliable way. But even more, you need to make a storage interface that doesn't suck. I am currently working with a homegrown S3 "like" interface. It is a massive pain in the arse as it doesn't scale[1], has an entirely new set of words to describe standard things (trying to overwrite a file? it doesn't tell you that, it just says "predicate failed") [1] large files transfer very fast, faster than s3. However all the tools are written single threaded. Add to this that each operation takes at least .7-1.5 seconds, shit gets slow very quickly. There is little to no documentation, the API is odd. in short, much as it annoys me to say this: for general purpose storage, buy over build.

spookthesunset 5y ago

Is that stuff even stored on spinning rust these days? I mean solid state might be more expensive upfront but I imagine at scale the energy consumption required to power and cool millions of mechanical devices becomes a big issue. Not to mention the random access performance of spiny disks ain’t too great either.

discodave 5y ago
Yes, spinning rust is still used for low cost storage. You can look up Dropbox Magic Pocket and I think some stuff that AWS has made public about S3.