4 ms·
As mentioned in the article, it only affected recently written data of 'Google Compute Engine' services. GCE allows user to launch VMs and generate arbitrary da
by bsurmanski 11y ago
As mentioned in the article, it only affected recently written data of 'Google Compute Engine' services. GCE allows user to launch VMs and generate arbitrary data on the server.
Normally, Google redundantly distributes out data to at least 3 different geographically distinct locations. Check out the 'BigTable' white paper [0] for more info.
For 99% of cases (and pretty well all user cases), this would not cause data lose. The key here is that the data was generated on the servers and did not have a chance to duplicate before the event.
[0] http://research.google.com/archive/bigtable-osdi06.pdf http://research.google.com/archive/bigtable-osdi06.pdf
- axiak 11y agoNowadays they use Reed–Solomon coding to effectively distribute their data without copying it to 3 places.
- Artemis2 11y agoDo you have a source for that? Why would they start using it now?
- magicalist 11y agoAlso how would that help with geographic redundancy? Local recovery from errors does you no good if your datacenter gets wiped out in a flood.
- azurezyq 11y agoDisclaimer: googler, but not working on storage. link here: http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.reverse-proxy.org/en/us/university/relations/facultysummit2010/storage_architecture_and_challenges.pdf http://static.googleusercontent.com/external_content/untrust... actually in colossus one can tune RS coding parameters per file, to get a tradeoff between performance/durablity. RS coding uses less copies, but same level of safety (tradeoff is the recovery computation time.)
- axiak 11y agoI can't find the presentation anywhere, but at HBaseCon 2014 one of the lead developers of Bigtable stated that they went to RS. Even for their older databases. EDIT: In this video https://vimeo.com/100153741 https://vimeo.com/100153741, around the 23 minute mark