4 ms·

Wouldn’t sharding generally result in overlap between the key ranges so that a database shard going down doesn’t have to result in any downtime? Then your issu

by codyb 6y ago

Wouldn’t sharding generally result in overlap between the key ranges so that a database shard going down doesn’t have to result in any downtime? Then your issue is replication of writes I suppose. Probably depends on use case what configuration you choose.

Tostino 6y ago
Totally depends on how you shard. In my case (b2b), i'd be sharding by tenant. Having a single tenant go down would not have the same impact as every single tenant going down.
evanelias 6y ago
With a typical sharded relational database setup, each sharding key value maps to exactly one shard. There should be replicas of that shard, which can be promoted in case of a master failure. But in rare cases all replicas may also be degraded or inconsistent and therefore non-promotable. When this happens to a giant monolithic non-sharded database, the impact is far more catastrophic than when it happens to a single smaller shard. In any case, replication is a separate concern from sharding. Each sharded database replica set (traditionally 1 master + N replicas) has a data set that is unique / independent of other sharded database replica sets. That said, some of the "NewSQL" distributed databases may arrange their data in more complex ways. [Edit to clarify: I mean in respect to the combination of both sharding and replication. With a traditional sharded relational database, you have replica sets where each replica contains the exact same shard or shards; with a NewSQL distributed DB, data may be replicated in a more complex arrangement]
- zinclozenge 6y ago
  Most, if not all, commercial newsql distributed databases do range based splitting of the data, with each range managed by a raft group. Raft groups get migrated between nodes to avoid hot spots, among other scheduling criteria. TiDB does this for sure, I'd be surprised if CockroachDB (and yugabyte and dgraph) doesn't do it.
wejick 6y ago
Honestly I never found the case when this happens, data always falls into 1 shard according to the key. Then comes the concept of shard replica where the shard can live in several nodes and form a redundancy. However I'm noy sure how usually it's being setup on Postgres

by codyb 6y ago

Tostino 6y ago

Totally depends on how you shard. In my case (b2b), i'd be sharding by tenant. Having a single tenant go down would not have the same impact as every single tenant going down.

evanelias 6y ago

With a typical sharded relational database setup, each sharding key value maps to exactly one shard. There should be replicas of that shard, which can be promoted in case of a master failure. But in rare cases all replicas may also be degraded or inconsistent and therefore non-promotable. When this happens to a giant monolithic non-sharded database, the impact is far more catastrophic than when it happens to a single smaller shard. In any case, replication is a separate concern from sharding. Each sharded database replica set (traditionally 1 master + N replicas) has a data set that is unique / independent of other sharded database replica sets. That said, some of the "NewSQL" distributed databases may arrange their data in more complex ways. [Edit to clarify: I mean in respect to the combination of both sharding and replication. With a traditional sharded relational database, you have replica sets where each replica contains the exact same shard or shards; with a NewSQL distributed DB, data may be replicated in a more complex arrangement]

zinclozenge 6y ago
Most, if not all, commercial newsql distributed databases do range based splitting of the data, with each range managed by a raft group. Raft groups get migrated between nodes to avoid hot spots, among other scheduling criteria. TiDB does this for sure, I'd be surprised if CockroachDB (and yugabyte and dgraph) doesn't do it.

wejick 6y ago

Honestly I never found the case when this happens, data always falls into 1 shard according to the key. Then comes the concept of shard replica where the shard can live in several nodes and form a redundancy. However I'm noy sure how usually it's being setup on Postgres