5 ms·

There is a new generation of relational databases that are native to multi-node operation, and don't require sharding. I'm speaking of tech like Google Spanner

There is a new generation of relational databases that are native to multi-node operation, and don't require sharding. I'm speaking of tech like Google Spanner and my employer, FaunaDB. Now you don't have to shard. More info on how we accomplish distributed transactions. https://fauna.com/blog/distributed-consistency-at-scale-spanner-vs-calvin https://fauna.com/blog/distributed-consistency-at-scale-span...

brianwawok 9y ago
P.S. Google Spanner and FaunaDB both shard. They can call it something else. But unless every node has all data on it, it is sharded.
- freels 9y ago
  It is true that Spanner and FaunaDB partition a cluster's dataset across multiple nodes but it's handled transparently by the database. Whenever I've heard the term "sharding" it's usually in reference to the application-level sharding described in the article. Partitioning the dataset isn't really novel these days (Cassandra, Riak, Mongo et al do the same of course), but what is a significant difference is that both Spanner and FaunaDB implement ACID transactions distributed across partitions. It no longer matters for application correctness what partition key you choose if you can involve any arbitrary set of records in an single transaction.
  - ozgune 9y ago
    (Ozgun from Citus Data) > Whenever I've heard the term "sharding" it's usually in reference to the application-level sharding described in the article. I wanted to drop a quick clarification note here. In the article, I used the term "sharding" to refer to both application and database level sharding. For anyone that's looking at sharding as an option for scaling, we're always happy to chat and help point you in the right direction. My email's ozgun @ citusdata.com If you're looking databases that come with built-in sharding, I'd definitely check out Citus (then again, I'm biased): https://www.citusdata.com/ https://www.citusdata.com/
  - zaptheimpaler 9y ago
    Abstractions are leaky. As soon as you run into a query that runs fast when the data is on one shard and slow when its not, you now need to know about sharding.
dboreham 9y ago
_You_ don't have to, but someone does.
- novembermike 9y ago
  Right, but it's the default state of the database.
berns 9y ago
The title tag of Spanner home page reads: Cloud Spanner | Automatic Sharding with Transactional Consistency at Scale

brianwawok 9y ago

P.S. Google Spanner and FaunaDB both shard. They can call it something else. But unless every node has all data on it, it is sharded.

freels 9y ago
It is true that Spanner and FaunaDB partition a cluster's dataset across multiple nodes but it's handled transparently by the database. Whenever I've heard the term "sharding" it's usually in reference to the application-level sharding described in the article. Partitioning the dataset isn't really novel these days (Cassandra, Riak, Mongo et al do the same of course), but what is a significant difference is that both Spanner and FaunaDB implement ACID transactions distributed across partitions. It no longer matters for application correctness what partition key you choose if you can involve any arbitrary set of records in an single transaction.
- ozgune 9y ago
  (Ozgun from Citus Data) > Whenever I've heard the term "sharding" it's usually in reference to the application-level sharding described in the article. I wanted to drop a quick clarification note here. In the article, I used the term "sharding" to refer to both application and database level sharding. For anyone that's looking at sharding as an option for scaling, we're always happy to chat and help point you in the right direction. My email's ozgun @ citusdata.com If you're looking databases that come with built-in sharding, I'd definitely check out Citus (then again, I'm biased): https://www.citusdata.com/ https://www.citusdata.com/
- zaptheimpaler 9y ago
  Abstractions are leaky. As soon as you run into a query that runs fast when the data is on one shard and slow when its not, you now need to know about sharding.

dboreham 9y ago

_You_ don't have to, but someone does.

novembermike 9y ago
Right, but it's the default state of the database.

berns 9y ago

The title tag of Spanner home page reads: Cloud Spanner | Automatic Sharding with Transactional Consistency at Scale