5 ms·

It would cost about US$2.5 million/year to back it up using Amazon Glacier.

by dmd 12y ago

LeoPanthera 12y ago
With that much data, it would be cheaper to build your own Glacier-like service.
- lunixbochs 12y ago
  About $1m to build 21 PB of live storage (not tape), and $50k/year to power it (not factoring maintenance, rack space cost, and cooling) A service offering live storage at this (Petabyte) scale [1] - costs 22,020,096 GB * 3¢/GB * 12mo = $7,927,234.56 a year. You could build six fully-mirrored copies for about that much, though management of 360 4U machines (30x 48U full-height racks worth) is a business on its own (see BackBlaze photos), and you need to deal with drive replacement after a point. [1] http://www.rsync.net/products/petabyte.html http://www.rsync.net/products/petabyte.html (If you actually sign up for this, be sure to send me that $24,000 referral bonus o_o) Math: -- Machine Cost -- $300 * 45 * 8TB drives = $13,500 (drive cost) + $3,387.28 (pod cost) == $16,887.28 (unit cost) / 360TB = $46.91 per TB $46.91/TB * (21 PB * 1024 == 21,504 TB) == $1,008,752.64 / 21 PB (for no redundancy) -- Power Cost -- Let's say the motherboard/controllers draw around 100W continuous and each drive pulls 11W (from my post elsewhere in the comments). This is 2,688 drives, which fit into 60x 45-drive 4U BackBlaze storage pods. 2,688 * 11W = 29,568 W (for the drives) + 6,000 W (for the computers) = 35.568 kW to run a non-redundant 21 PB array. You'll need at least two network switches and moderate cooling infrastructure as well. The heat from using 35.5 kWh continuously takes nontrivial effort to disperse, but let's assume that's somehow free for the sake of simplicity here. 35.568 kW * 24 * 365 = 311.57568 MWh (in a year) I'm sure at this point you might be able to work out a bulk deal of some kind, but at $0.16/kWh you're talking a pretty low cost of $49,852.11 in power every year. ---- Building arrays of nonredundant live drives would be significantly cheaper over the long term than Amazon Glacier (which is almost certainly cold storage, not live) at this scale.
  - cshimmin 12y ago
    Right but Glacier _will_ provide redundancy. If you introduce even the most basic redundancy to your math (multiply everything by 2), it pretty nearly closes the gap with Glacier's costs. You also forgot to add in the costs of paying at least one human (probably a few) to set all this up and maintain it forever. The real question is, why do they need this archive to be spinning? What's wrong with tape?
  - ghaff 12y ago
    >Amazon Glacier (which is tape-based!) Citation? Not saying you're wrong but I've seen supposedly "in the know" commentary in favor of both the tape-based and some sort of cold disk-based storage. I've been curious and also a bit surprised that the true answer has never really hit publicly although I'm sure it's well-known within some communities.
    - lunixbochs 12y ago
      Oh, you're right. I haven't thought about this for a while, and I remember my previous assumption about tape storage being challenged. I'll replace that part with "cold storage". After refreshing my memory: Amazon outright they're using denied tape backup, some third-parties pointed out cold spinning disks wouldn't be very economical, and a few people theorized it's actually BDXL.
      - ghaff 12y ago
        Cold storage is probably safe. I've seen the assertion that it's tape from someone who I wouldn't have thought would get it wrong but I'd now bet against it whether or not Amazon's denial seals the deal. Robin Harris has argued in favor of BDXL but he's speculated about a number of things. http://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ http://storagemojo.com/2014/04/25/amazons-glacier-secret-bdx... My money would be on the simplest: cold disk. Some support here: http://mjtsai.com/blog/2014/04/27/how-is-amazon-glacier-implemented/ http://mjtsai.com/blog/2014/04/27/how-is-amazon-glacier-impl... but it's all speculative and anecdotal. I don't have a problem with the thought that Amazon is getting out ahead of the drive economics, especially given that I'm not sure how widely used Glacier is today. I do find it interesting that this has been apparently kept under wraps from anyone who would share more widely.

It would cost about US$2.5 million/year to back it up using Amazon Glacier.

by dmd 12y ago

It would cost about US$2.5 million/year to back it up using Amazon Glacier.

LeoPanthera 12y ago

With that much data, it would be cheaper to build your own Glacier-like service.

lunixbochs 12y ago
About $1m to build 21 PB of live storage (not tape), and $50k/year to power it (not factoring maintenance, rack space cost, and cooling) A service offering live storage at this (Petabyte) scale [1] - costs 22,020,096 GB * 3¢/GB * 12mo = $7,927,234.56 a year. You could build six fully-mirrored copies for about that much, though management of 360 4U machines (30x 48U full-height racks worth) is a business on its own (see BackBlaze photos), and you need to deal with drive replacement after a point. [1] http://www.rsync.net/products/petabyte.html http://www.rsync.net/products/petabyte.html (If you actually sign up for this, be sure to send me that $24,000 referral bonus o_o) Math: -- Machine Cost -- $300 * 45 * 8TB drives = $13,500 (drive cost) + $3,387.28 (pod cost) == $16,887.28 (unit cost) / 360TB = $46.91 per TB $46.91/TB * (21 PB * 1024 == 21,504 TB) == $1,008,752.64 / 21 PB (for no redundancy) -- Power Cost -- Let's say the motherboard/controllers draw around 100W continuous and each drive pulls 11W (from my post elsewhere in the comments). This is 2,688 drives, which fit into 60x 45-drive 4U BackBlaze storage pods. 2,688 * 11W = 29,568 W (for the drives) + 6,000 W (for the computers) = 35.568 kW to run a non-redundant 21 PB array. You'll need at least two network switches and moderate cooling infrastructure as well. The heat from using 35.5 kWh continuously takes nontrivial effort to disperse, but let's assume that's somehow free for the sake of simplicity here. 35.568 kW * 24 * 365 = 311.57568 MWh (in a year) I'm sure at this point you might be able to work out a bulk deal of some kind, but at $0.16/kWh you're talking a pretty low cost of $49,852.11 in power every year. ---- Building arrays of nonredundant live drives would be significantly cheaper over the long term than Amazon Glacier (which is almost certainly cold storage, not live) at this scale.
- cshimmin 12y ago
  Right but Glacier _will_ provide redundancy. If you introduce even the most basic redundancy to your math (multiply everything by 2), it pretty nearly closes the gap with Glacier's costs. You also forgot to add in the costs of paying at least one human (probably a few) to set all this up and maintain it forever. The real question is, why do they need this archive to be spinning? What's wrong with tape?
- ghaff 12y ago
  >Amazon Glacier (which is tape-based!) Citation? Not saying you're wrong but I've seen supposedly "in the know" commentary in favor of both the tape-based and some sort of cold disk-based storage. I've been curious and also a bit surprised that the true answer has never really hit publicly although I'm sure it's well-known within some communities.
  - lunixbochs 12y ago
    Oh, you're right. I haven't thought about this for a while, and I remember my previous assumption about tape storage being challenged. I'll replace that part with "cold storage". After refreshing my memory: Amazon outright they're using denied tape backup, some third-parties pointed out cold spinning disks wouldn't be very economical, and a few people theorized it's actually BDXL.
    - ghaff 12y ago
      Cold storage is probably safe. I've seen the assertion that it's tape from someone who I wouldn't have thought would get it wrong but I'd now bet against it whether or not Amazon's denial seals the deal. Robin Harris has argued in favor of BDXL but he's speculated about a number of things. http://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ http://storagemojo.com/2014/04/25/amazons-glacier-secret-bdx... My money would be on the simplest: cold disk. Some support here: http://mjtsai.com/blog/2014/04/27/how-is-amazon-glacier-implemented/ http://mjtsai.com/blog/2014/04/27/how-is-amazon-glacier-impl... but it's all speculative and anecdotal. I don't have a problem with the thought that Amazon is getting out ahead of the drive economics, especially given that I'm not sure how widely used Glacier is today. I do find it interesting that this has been apparently kept under wraps from anyone who would share more widely.