4 ms·
Those are fundamentally different things. Sharding is a scaling strategy for reads, writes and data size. It doesn't address durability (which many distributed
by mjb 11y ago
Those are fundamentally different things. Sharding is a scaling strategy for reads, writes and data size. It doesn't address durability (which many distributed databases do). It doesn't address the discoverability problem of how you know which piece of data is where (easy if you have a well-distributed natural key, potentially difficult if you have a poorly distributed natural key, potentially obscenely difficult if you don't have a natural key). It doesn't address data locality (it can, but then you need to migrate keys, and end up with a second-order 'which key is where?' problem). Most crucially, it doesn't address what you need to do if you have to do transactions (of any kind) across shards. You can try invent something to do that. You may get it right (probably using consensus or 2PC), or you may get it fundamentally wrong. Either way, it's a tricky problem.
Sharding is good, for sure. It's not a replacement for this kind of technology.
Aphyr is doing great work keeping vendors honest, and making sure they live up to their marketing. I don't think any of this says that clustered databases are inherently shit.
- tedunangst 11y agoThat's fair. I may have been a little cavalier with the use of the word shit. Though I note auto sharding is a headline feature for many systems tested. And as for durability, many seem to fall short of rsync running out of cron.