3 ms·
Sharding is generally used to refer to a particular type of partitioning: physically distributing data on different machines with the purpose of scaling horizo
by gregwebs 8y ago
Sharding is generally used to refer to a particular type of partitioning: physically distributing data on different machines with the purpose of scaling horizontally (to increase write capacity).
What many NewSQL databases use is a logical partitioning scheme where data is partitioned into <= 1 GB chunks. This makes it easy to both scale horizontally and achieve fault tolerance (and change these characteristics without interrupting service).
In some systems that component may be coupled and hidden away. In TiDB it is a separately deployed component called the Placement Driver (which embeds etcd, a Chubby equivalent): https://pingcap.com/blog/2016-11-09-Deep-Dive-into-TiKV/#placement-driver https://pingcap.com/blog/2016-11-09-Deep-Dive-into-TiKV/#pla...
With sharding you do still need a system to locate your data. A big difference is that most sharding systems are not designed to distribute transactions (or possibly even joins) across shards. So Vitess has a special 2PC for cross-shard transactions with poorer performance than a single shard transaction.