4 ms·
Does this perform Paxos on every single transaction (wouldn't that be expensive?) or is Paxos used to elect a write coordinator replica and ensure a consistent
by strlen 16y ago
Does this perform Paxos on every single transaction (wouldn't that be expensive?) or is Paxos used to elect a write coordinator replica and ensure a consistent view of the cluster to guarantee serializable consistency?
If it's former, can we get more details? This would be quite similar to the work Daniel Abadi had been doing at Yale at parallelizing distributed commit (e.g., Paxos) to maintain high throughput despite increased latency.
- rbranson 16y agoI'm going to assume the later. Why would they need to perform Paxos on every single transaction when just using it to elect a master gets 99.9% of the benefits? That would be consistent with the implementation of BigTable in general. EDIT: The higher write latency and 3x CPU cost is probably because it uses synchronous replication to write to all of the slaves before acknowledging the transaction.
- strlen 16y ago> I'm going to assume the later. Why would they need to perform Paxos on every I'm going to assume the later. Why would they need to perform Paxos on every single transaction when just using it to elect a master gets 99.9% of the benefits? That would be consistent with the implementation of BigTable in general. That's my thought as well: essentially they have a bigtable cluster replicated in multiple datacenters, with WALs being shipped between them. That's consistent with the way Google builds systems and with recent papers suggests that correlated failures are more common than one thinks and going to a different datacenter is a wiser strategy than increasing replication factor within a single datacenter.
- powera 16y agoA reasonable thought, but wrong. The existing datastore has basically the 99.9% of the benefits you describe. Unfortunately, 99.9% availability (especially when a single datastore request may touch 10 things and thus be at 99% availability) isn't good enough. One of the major problems with having a single bigtable is partial outages. These can either be the major events like there were today, with 10% or more of the cell having issues, or it can be just one server in the bigtable having performance issues. The key to having higher availability with the High Replication datastore is that if certain entities are slow or unavailable in one datacenter, we can rely on the performance of other datacenters to limit the latency.
- strlen 16y agoIs this the first production use of Spanner? (No worries if you're not allowed to comment on this, sorry to be nosy, but this is something I'm curious about)
- rbranson 16y agoThanks for the clarification, but I'm not sure I follow what I was wrong about. I wasn't saying that it should have 99.9% availability, I was just stating that in most situations, using Paxos to elect a master provides very nearly all the benefits (99.9% was just a vague figure to indicate this) of doing a Paxos commit each time.