4 ms·

AWS Glacier provides a pretty cheap[0] backend for such a service. You could store 1TB for 1000GB * $0.004/(GB * mo) = $4.00/mo, which is dirt cheap, all things

AWS Glacier provides a pretty cheap[0] backend for such a service. You could store 1TB for 1000GB * $0.004/(GB * mo) = $4.00/mo, which is dirt cheap, all things considered. Retrieval is about 10x that, per GB, but generally if you are storing backups, you aren't concerned about retrieving it too often, so that cost is acceptable. Compare this to the S3 bucket costs for the same storage size($10/mo for infrequent access, and $24/mo for standard), or to google drive 1TB tier ($10/mo, but no API as far as I know). [0] https://aws.amazon.com/glacier/pricing/ https://aws.amazon.com/glacier/pricing/

oliwarner 8y ago
That's per month... In just 9 months Glacier becomes more expensive than a physical SATA drive(s) of equal capacity. The longer you keep the data, the more it skews against "the cloud". I'm definitely not claiming to have all the answers here but there are points where tiering your storage strategy to get it onto something stable and without running costs makes sense.
- mmt 8y ago
  Indeed, at 1TB scale, cloud storage can sound "dirt cheap", and, of course, it is, since the fixed cost of hardware to store that reliably is the same as for 100TB. 10x that, or a petabyte, would cost $144k-$180k after 3 years, depending on region. Hardware [1] might be, generously, $60k. The hardware would also have operating costs of another $26k [2], but Glacier would have retrieval and data transfer costs if one ever wanted the data back. Of course, tape, the subject of TFA, is even better. $17k [3] for the hardware and minimal operating cost in the labor required to change the tapes. VC-funded startups have been tending to ignore this artithmetic (which applies to all cloud infrastructure, at only slightly larger scales), lately, so cost isn't always an issue any more. [1] Let's say 100 12TB drives for 84 data, 12 (triple) parity, 4 warm spares. 3 enclosures, 6 SAS HBAs, plenty of CPU and RAM for ZFS. [2] Generously, the above is 2.7kW. Adding 50% for cooling, at 12c/kWh. A commercial datacenter could double that (or triple or more, but, for backups, presumably one would find a geographically cheaper one). [3] LTO-8 drives are $2700 and 12TB cartridge are $170.
  - mmt 8y ago
    > Of course, tape, the subject of TFA, is even better. $17k > LTO-8 drives are $2700 and 12TB cartridge are $170. What I initially ignored was the cost of a "library" (robot), which is, arguably, necessary if the minimum time to write a tape is over 9 hours (longer than a standard workday, meaning 1PB could take 17 workweeks to write with a single drive with no robot). Including a library to hold all 84 cartridges might be tricky, but it appears there are 80-slot ones in the $12k range, which brings the total tape price up to $29k and reduces the write time to under 5 weeks. Alternatively, one could just buy multiple tape drives and no robot. $25k gets faster write time than above.

oliwarner 8y ago

That's per month... In just 9 months Glacier becomes more expensive than a physical SATA drive(s) of equal capacity. The longer you keep the data, the more it skews against "the cloud". I'm definitely not claiming to have all the answers here but there are points where tiering your storage strategy to get it onto something stable and without running costs makes sense.

mmt 8y ago
Indeed, at 1TB scale, cloud storage can sound "dirt cheap", and, of course, it is, since the fixed cost of hardware to store that reliably is the same as for 100TB. 10x that, or a petabyte, would cost $144k-$180k after 3 years, depending on region. Hardware [1] might be, generously, $60k. The hardware would also have operating costs of another $26k [2], but Glacier would have retrieval and data transfer costs if one ever wanted the data back. Of course, tape, the subject of TFA, is even better. $17k [3] for the hardware and minimal operating cost in the labor required to change the tapes. VC-funded startups have been tending to ignore this artithmetic (which applies to all cloud infrastructure, at only slightly larger scales), lately, so cost isn't always an issue any more. [1] Let's say 100 12TB drives for 84 data, 12 (triple) parity, 4 warm spares. 3 enclosures, 6 SAS HBAs, plenty of CPU and RAM for ZFS. [2] Generously, the above is 2.7kW. Adding 50% for cooling, at 12c/kWh. A commercial datacenter could double that (or triple or more, but, for backups, presumably one would find a geographically cheaper one). [3] LTO-8 drives are $2700 and 12TB cartridge are $170.
- mmt 8y ago
  > Of course, tape, the subject of TFA, is even better. $17k > LTO-8 drives are $2700 and 12TB cartridge are $170. What I initially ignored was the cost of a "library" (robot), which is, arguably, necessary if the minimum time to write a tape is over 9 hours (longer than a standard workday, meaning 1PB could take 17 workweeks to write with a single drive with no robot). Including a library to hold all 84 cartridges might be tricky, but it appears there are 80-slot ones in the $12k range, which brings the total tape price up to $29k and reduces the write time to under 5 weeks. Alternatively, one could just buy multiple tape drives and no robot. $25k gets faster write time than above.