2 ms·
> - Determining which database each piece of data lives on (as opposed to using partitioning keys which do that automatically) Most sharding databases use cons
by itake 2y ago
> - Determining which database each piece of data lives on (as opposed to using partitioning keys which do that automatically)
Most sharding databases use consistent hashing.
> - Manually rebalancing data, which is often difficult and error-prone
not really. When you setup the database, you choose a highly divisible number of shards and then consistent hashing to spread the data across the shards. Each server hosts N-shards where N changes as your data (and server count) grows
> - Adding partitions manually as the system grows
Not really. Just choose a reasonably high number of shards and divide them across your servers.
> - (Anecdotally) Higher operational costs, since matching node count to workload is tricky
This could be true, but also, there is overhead to managing a ton of SQLite databases too. I think there are tradeoffs here.