4 ms·

Your cost estimates are outlandish, making Amazon S3 look like a bargain. In reality 1.4PB raw will set you back well below $50k/pa, not $239k/pa. If you're l

by moe 12y ago

Your cost estimates are outlandish, making Amazon S3 look like a bargain. In reality 1.4PB raw will set you back well below $50k/pa, not $239k/pa. If you're lazy you could even rent 1.4PB raw from e.g. Hetzner and you'd still be paying under $90k/pa (that would be 23 of their SX290 servers).

skuhn 12y ago
You can do significantly better with the right vendors and some negotiation -- this is specifically a pricing ballpark, not rock bottom deals. It's not outlandish at all to say that in terms of raw storage you're going to be close to S3's pricing, but it also doesn't reflect well on S3 at all. They are buying at a scale far beyond what you can hope to achieve. This is money spent on physical equipment that you can choose to lease, that you can take depreciation on, that you can resell -- these are assets, not a line item on a monthly bill. S3 has no such benefits. And this doesn't even taking into account bandwidth, which is the real killer. The stored data has to come out sometime. 1.4PB of raw storage requires ~374 4TB hard drives. If you want to spend $50,000, you have at most $130 to spend on drives, and that's only if you get literally everything else for free including electricity. You can almost buy 4TB desktop class drives for that price from Newegg, but not quite. Don't buy drives from retail channels, and don't buy desktop drives at all (yes, I know Backblaze does both of these things). HGST 4TB MegaScale or UltraStar drives are a safe bet, depending on your performance requirements. They run between $250-300 in quantity, which means 1.4PB is $93,500 in disks alone.
- moe 12y ago
  Stop switching around between hardware cost and yearly cost, jesus... (pa means per annum) 0.7PB on Amazon S3 costs $810k for 3 years (as per their cost calculator). Renting 1.4PB raw from Hetzner costs about $270k for 3 years. Running 1.4PB raw on your own hardware costs about $150k for 3 years (on Supermicro JBODs). It's not that hard really, and my point merely was that the difference is much bigger than you made it sound.
  - parasubvert 12y ago
    Apples, to apples please. 0.7PB on Amazon S3 reduced redundancy is $582,660 over three years with geo-redundancy and availability across an entire region, and lifecycle to push to Glacier which would drop the price to $289,872 annually given 50,000 monthly uploads and 20% monthly retrieval. No labour when it comes to dealing with equipment failures. I'd also say that only SMBs and web/internet companies run the equivalent SuperMicro JBODs, most large companies use enterprise storage vendors like EMC, NetApp, HP/3PAR which are premium priced.
    - moe 12y ago
      Apples, to apples please. You are right, I should have compared to S3 reduced redundancy, sorry. So we're down to $582k (S3-RR) vs $270k (Hetzner) vs $150k (own hardware). That's still a healthy 2x-4x difference. Glacier If you introduce Glacier then it's not apples-to-apples anymore.
      - parasubvert 12y ago
        "If you introduce Glacier then it's not apples-to-apples anymore." This is a thread about storage costs for backup as a service, so I felt at least it's in the same family of tree-bearing fruit ;)

by moe 12y ago

skuhn 12y ago

You can do significantly better with the right vendors and some negotiation -- this is specifically a pricing ballpark, not rock bottom deals. It's not outlandish at all to say that in terms of raw storage you're going to be close to S3's pricing, but it also doesn't reflect well on S3 at all. They are buying at a scale far beyond what you can hope to achieve. This is money spent on physical equipment that you can choose to lease, that you can take depreciation on, that you can resell -- these are assets, not a line item on a monthly bill. S3 has no such benefits. And this doesn't even taking into account bandwidth, which is the real killer. The stored data has to come out sometime. 1.4PB of raw storage requires ~374 4TB hard drives. If you want to spend $50,000, you have at most $130 to spend on drives, and that's only if you get literally everything else for free including electricity. You can almost buy 4TB desktop class drives for that price from Newegg, but not quite. Don't buy drives from retail channels, and don't buy desktop drives at all (yes, I know Backblaze does both of these things). HGST 4TB MegaScale or UltraStar drives are a safe bet, depending on your performance requirements. They run between $250-300 in quantity, which means 1.4PB is $93,500 in disks alone.

moe 12y ago
Stop switching around between hardware cost and yearly cost, jesus... (pa means per annum) 0.7PB on Amazon S3 costs $810k for 3 years (as per their cost calculator). Renting 1.4PB raw from Hetzner costs about $270k for 3 years. Running 1.4PB raw on your own hardware costs about $150k for 3 years (on Supermicro JBODs). It's not that hard really, and my point merely was that the difference is much bigger than you made it sound.
- parasubvert 12y ago
  Apples, to apples please. 0.7PB on Amazon S3 reduced redundancy is $582,660 over three years with geo-redundancy and availability across an entire region, and lifecycle to push to Glacier which would drop the price to $289,872 annually given 50,000 monthly uploads and 20% monthly retrieval. No labour when it comes to dealing with equipment failures. I'd also say that only SMBs and web/internet companies run the equivalent SuperMicro JBODs, most large companies use enterprise storage vendors like EMC, NetApp, HP/3PAR which are premium priced.
  - moe 12y ago
    Apples, to apples please. You are right, I should have compared to S3 reduced redundancy, sorry. So we're down to $582k (S3-RR) vs $270k (Hetzner) vs $150k (own hardware). That's still a healthy 2x-4x difference. Glacier If you introduce Glacier then it's not apples-to-apples anymore.
    - parasubvert 12y ago
      "If you introduce Glacier then it's not apples-to-apples anymore." This is a thread about storage costs for backup as a service, so I felt at least it's in the same family of tree-bearing fruit ;)