4 ms·
As you have already discovered, storing that data on S3 is rather expensive. S3 is priced in a way that makes sense for hot, important, accessible data; not for
by kylecordes 16y ago
As you have already discovered, storing that data on S3 is rather expensive. S3 is priced in a way that makes sense for hot, important, accessible data; not for idle permanent archives.
I attacked a problem like this a few years ago, and found that the best solution at the time was to build servers with lots of big cheap hard drives and as little else as possible; similar to the BackBlaze idea but less impressive. I wrote it up here:
http://kylecordes.com/2008/big-raid http://kylecordes.com/2008/big-raid
...but since hardware changes so fast, all of the details there are probably obsolete.
Extra servers with backup copies of data can be much more economically deployed in an office closet instead of a colo. There will be more downtime (power and bandwidth reliability), but if it's an idle backup, that won't matter.
Servers full of hard drives (20 TB worth or whatever) for long-term, un-accessed archives can be turned off (no power or heat), but opinions vary as to whether the on/off cycles for occasional access will reduce the total service life.