7 ms·
Facebook's new optical storage solution [video]
- drcross 13y agoNo comment on the access time and having only one drive means reading only one disc at once. This puts this storage solution in the realm of "I don't need this data within an hour" so I presume the application this system acts as redundancy for failed disks.
- baddox 13y agoThe guy was very explicit that this is a cold storage solution. Probably "backups of backups."
- NatW 13y agoThe Facebook open computer definition for "cold storage": "where data is stored on disk but almost never read again, like legal data or backups of third copies of data — keep on increasing dramatically, there are huge demands for developing some form of cold storage system with the highest capacity and the lowest cost." source: http://www.opencompute.org/projects/storage/ http://www.opencompute.org/projects/storage/
- nivla 13y agoAre these Blu-ray disks rewritable? If so, wouldn't this be a great tech for Amazon's Glacier service? I believe they currently depend of costly tape drives to store data.
- dsl 13y agoWhy bother with rewriteable disks? This is cold storage meaning that they will likely never throw this data away. I'd suspect Amazon's Glacier runs on something similar.
- nivla 13y ago>Why bother with rewriteable disks? Its okay for Facebook and other single entities for backups. However for Glacier, it would mean throwing away the disk whenever someone deletes their data. It would also means constant replacement of disks.
- ceejayoz 13y ago> However for Glacier, it would mean throwing away the disk whenever someone deletes their data. Probably not a huge concern, given the use case. Glacier is pretty fire-and-forget. > It would also means constant replacement of disks. They're already doing that with servers, why would disks be a problem?
- bluthru 13y agoDo rewritable blurays last as long as write-once discs?
- akg_67 13y agoEach LTO-6 tape cartridge stores 6TB of data. Each quadruple layer Blue-ray disk stores only 128GB. I am surprised that FB went with Optical Disk instead of Tape. Optical jukeboxes/libraries for MO/DVD/CD used to be popular a decade or so ago and now considered antique. There is just not enough density with optical.
- mrtksn 13y agohow reliable are these tapes?
- sargun 13y agoMy guess: 1) Lower cost for drives (commodity drives). 2) The ability to (easily) seek to an arbitrary location on disk 3) Cost of BD burners are <$100; Cost per GB of disks is $0.025/GB. LTO-6 drives are $1000+, and cost per GB is around $0.012/GB. -- I think the cost works out actually, especially as the cost of blu-ray is still falling.
- deleted 13y ago[deleted]
- mcpherrinm 13y agoI believe Glacier is an all-hard-drive solution. Tapes are troublesome because they're not "online" so it's harder to verify data integrity and notice failures. The drives are spun much slower than a regular drive (I've heard 300rpm) and so they can be crammed into much higher density racks without worrying about heat. Maybe data is eventually moved to tapes for Glacier; I am privy to no special information.
- fastball 13y agoMy hopes were dashed when I realized that this wasn't a new type of optical hard disc, like holographic drive technology or something.
- ricardobeat 13y agoMaybe in a couple years: http://rt.com/news/5d-nanostructured-glass-optical-memory-941/ http://rt.com/news/5d-nanostructured-glass-optical-memory-94...
- ksec 13y agoWith Big Data Everything, we really need an Optical Storage System like Blu Ray but with 100x to 1000x the capacity.
- dsl 13y agoFacebook has built a novel implementation, but this isn't a new idea or product. Similar to how they used to buy servers and now build them to spec, they probably used to buy these and now design and build them. http://www.disc-group.com/products/bd-series-benefits/ http://www.disc-group.com/products/bd-series-benefits/
- gatehouse 13y agoAfter I saw this on James Hamilton's blog the other day I did a quick check of BD-R versus 4TB HDDs cost per GB. Using whatever I could find after about 5 minutes of looking around, I came up with $0.05/GB for both types of media. If anyone can provide a link to BD media for sale at a price that make this economical I'd be interested to know.
- glomph 13y agoThe saving is in power consumption for cold storage.
- NatW 13y agoI would think the media's longevity factors into the lifetime cost. If the optical media is certified for 50+ years, that essentially means they won't have a need to recopy data if for some reason they want to preserve it.. for a long time.. I don't know how long magnetic tape lasts, but I would imagine it would be a huge pain to need to keep backing up old tapes to new tapes every X years.
- crazygringo 13y agoCan anyone knowledgeable explain why they use Blu-Ray instead of tape backup? When would an organization use Blu-Ray, and when wouldn't then? Is this reflective of any kind of general trend?
- jamesaguilar 13y agoContrary to the other two commenters, my guess was going to be that they are optimizing TB/m^3.
- kitcar 13y agoI'm not knowledgeable, but some total guesses: 1) As blu-ray is a consumer product its probably manufactured in greater quantities than backup grade tapes, and therefore can be had lower in price, and there are more vendors to source them from. Similarly the drives themselves are cheaper. 2) Tapes aren't random access, therefore it's slow and cumbersome to retrieve single files.
- jonny2112 13y ago3) IIRC most tape storage technolgies are closed source. FB has been heavily pushing their open compute project. Using BDs get them past a lot of licensing problems.
- jchendy 13y agoLTO is an open standard: http://en.wikipedia.org/wiki/Linear_Tape_Open http://en.wikipedia.org/wiki/Linear_Tape_Open
- wmf 13y agoI don't think LTO is any more proprietary than Blu-ray.
- protomyth 13y agoand LTO has way less licensing problems than Blu-ray and LTO-6 has a max of 2.5TB. LTO is also guaranteed to to read tapes from 2 generations earlier and write one generation earlier. Blu-ray has no such long term guarantee.
- rld 13y agoAt 1:55 he says "FRID'd", he meant RFID'd, right?
- kraemate 13y agoDoes anyone know what data is Facebook trying to archive? Old user posts? But that doesn't fit this usecase because of the high latency. Or maybe its just a nice content delivery mechanism to the NSA. Just ship a bunch of blue-rays instead of pesky optical fiber interception.
- jonny2112 13y agoFB is required by law to store all its credit card transactions that every singal FB ads account has for 6 years. (So are other US based companies that process credit cards)
- kraemate 13y agoSurely, credit-card info can't span 100s of terabytes, even if they record each and every transaction.
- shawabawa3 13y agobackups of images/videos/posts/everything
- treelovinhippie 13y agoAny guess on the total storage capacity in that rack and comparison to a similar sized magnetic storage?
- hornetblack 13y agoA comment on the video suggested 950 TB per rack. > 1 Rack = 44 magazines; 1 magazine = 36 cartridges; 1 cartridge = 12 blueray discs. So, ~950TB.
- nuclear_eclipse 13y agoFrom the video: 24 magazines/rack * 36 cartridges/magazine * 12 discs/cartridge * 50 GB/disc = 518TB/rack. Based standard dual-layer discs. I cannot comment on storage of our traditional spinning-disk racks.
- derekp7 13y agoBackblaze Pods are about 180TB, so 1800TB per rack. And a whole lot of power consumption.
- ryaneager 13y agoIf they use 128 GB Blu-Ray disks its about 1.26 PB per rack. 24 mag -> 36 cart -> 12 disks 10,368 Disks in total per rack
- imbriaco 13y agoI talked to them at the Open Compute Summit and got the chance to look at it up close. They quoted me 100GB/disc and 10,000 discs in the cabinet for 1PB. They could have been rounding those numbers for simplicity, but that gets you in the neighborhood.
- nicholasjarnold 13y agoAnother benefit of this system is that the storage media is immune electromagnetic pulse. So, even in the case of a large-scale EMP event our cat pictures and network graphs will remain intact and recoverable. :)
- MAGZine 13y agoTrue, but it's easy to build a storage container that doubles as a faraday cage when the tapes are in long-term storage. Actually, I'd be very surprised if this wasn't the case (pun intended).
- tlrobinson 13y agoI assume SSDs are screwed by EMPs, but how does magnetic media fare (the tape/disk media itself, not reader/writer electronics)?
- ricardobeat 13y agoWiped.
- chm 13y agoStill magnetic storage however you read/write the data...
- hsshah 13y agoFB: How can you get a consumer version of this so that you don't have to use Amazon Glacier for personal data archival?!
- polskibus 13y agoWhat about bit rot though? Some of my dvdr are no longer readable after 5 yrs, stored in very good conditions. Hdd backup, esp. with Zfs can provide very good bit rot protection.
- randartie 13y agoThese are high grade guaranteed for 30+ years. There's also 1000 year grade blu rays out there.
- polskibus 13y agoI had failures of both "archival" quality DVDs and SD cards before, just after a year or two after last backup. Do you have any links on the methodology used to measure the 30+ or 1000yr grade for a blu-ray ?
- x0054 13y agoBlu-rays are but a few years old. It's preposterous for any one to claim that they are "guaranteed" for 30 years, let alone 1,000 years. Not to mention that that guarantee most likely has an asterisk that limits damages to the cost of the original blu-ray disk in case it actually fails.
- callesgg 13y agoI was expecting some kind of optical fiber storage with reflecting crystals and shit. This was a bit disappointing.
- sneak 13y agoFlash required, so it won't play here. Really, Facebook? It's 2014.
- jpalomaki 13y agoI might be interested in cloud backup service which would backup my "keep forever" stuff, put them to optical disc and then mail the discs to me when they fill up. Could be also an add-on service to existing online backup business.
- cordite 13y agoI did not know that they made disks that are certified for long term storage like that.