3 ms·
What is the utility of cloud computing for archivists? Is it too expensive? It seems like the longevity is fairly proven at this point. If you put something in
by jeffbee 5y ago
What is the utility of cloud computing for archivists? Is it too expensive? It seems like the longevity is fairly proven at this point. If you put something in S3 in 2006, it's still there. GCS was launched over ten years ago, too. With a multi-cloud strategy you could probably make the case for durability, certainly as a 2nd or 3rd replica that just happens to be dramatically more convenient than optical discs.
Considering that many government records are now produced in the cloud anyway, it makes a certain sense. USGS elevation models for example are hosted on S3. Are they also on tape somewhere else?
- rbanffy 5y ago> Are they also on tape somewhere else? I can assure you they are and that there must be a number of copies available for use on any respectable supercomputer. It just may be that the canonical source of the data is, officially, the cloud and that they release the data by uploading it.
- KennyBlanken 5y agoIt is far too expensive due to the power. Remember that you're not keeping one drive per unit of data going, but multiple drives for redundancy. Then there's the power used by the fans, power supply losses, and then the datacenter cooling costs...plus all the network infrastructure and so on. Some of those are pretty small but it adds up to a lot to keep, say, 6TB work of data online vs in a LTO cartridge sitting on a shelf. Drive manufacturers, storage vendors, and cloud providers have been working on lowering opex/TCO for years. Bigger and bigger drives, helium which reduces aerodynamic drag, tiered storage where stuff is moved to slower and slower spindles the longer it goes without being accessed, for example...in some cases automatically getting pushed to tape. But you need systems to manage all that, systems that take power, have support contracts, and introduce a lot of complexity. Eventually flash memory will get cheap enough - but flash degrades with time, too.
- jeffbee 5y agoEh, but the whole point of the cloud is if you mix a cold storage workload on part of a disk and an active workload on another part of it then the marginal cost of the cold workload in terms of all that stuff you mentioned is almost zero.
- wmf 5y agohttps://blog.dshr.org/2019/02/cloud-for-preservation.html https://blog.dshr.org/2019/02/cloud-for-preservation.html
- selfhoster11 5y agoArchivists need access to this data decades, if not a century down the line. I trust the Internet Archive to be around somehow for much longer than S3.
- Kye 5y agoYep. Companies eventually die. Sometimes they're reborn, like IBM, AOL, and Microsoft, but never with the same priorities. The Internet Archive is almost as old as Amazon and archiving is its entire reason for being. The pendulum will swing back as it always has and send things off big [mainframes, clouds], and S3 will be sunset in favor of serving whatever comes next.