3 ms·

Everything has to be directed through a leader though. So it's not a matter of how close the client is to the cluster but how far the cluster's nodes are from e

• remram 4y ago
  In practice you don't run the consensus over the whole cluster for every shard of data. This would be inefficient in terms of data storage (every node stores a copy of everything) and latency. For every shard you pick a few nodes, and you run consensus between those. This is the way Elasticsearch, CockroachDB, ... work. So if a specific shard is getting writes from a specific region, you can easily move that shard to nodes in that region. In contrast, etcd runs consensus over the whole cluster. So it is very safe, very slow, and meant for small datasets. In many ways, it is not a database, but a metadata store for distributed systems.
  • eatonphil 4y ago
    Yup that makes sense. It wasn't clear from the article though how their system works and other sqlite+raft implementations do the simpler version of one raft cluster for the entire database, I think.
  • anonymousDan 4y ago
    This is not entirely accurate. If you want to forego the ability to support cross shard transactions then sure you can run consensus independently on each shard. But I don't think that is what Spanner/CockRoachDB do. In addition, even if you want to limit transactions to a a single shard, the shard replicas need to be in different datacenters/AZs if you want to remain available when a DC/AZ goes down.
    • deleted 4y ago
      [deleted]