4 ms·
I talk about these problems in the "How hard can sharding be?" section of the article — long story short, not all business requirements can be dealt with easily
by n2d4 10mo ago
I talk about these problems in the "How hard can sharding be?" section of the article — long story short, not all business requirements can be dealt with easily, but surprisingly many can if you choose a smart sharding key.
You can also still do optimistic concurrency across shards! That covers most of the remaining ones. Anything that requires anything more complex — sagas, 2PC, etc. — is relatively rare, and at scale, a traditional SQL OLTP will also struggle with those.
- qouteall 10mo agoThanks for reply. So in my understanding: - The transactions that only touch one shard is simple - The transactions that read multiple shards but only write shard can use simple optimistic concurrency control - The transactions that writes (and reads) multiple shards stay complex. Can be avoided by designing a smart sharding key. (hard to do if business requirement is complex)
- n2d4 10mo agoThat's right! If you anticipate you will encounter the third type a lot, and you don't anticipate that you will need to shard either way, what I'm talking about here makes no sense for you.
- qouteall 10mo agoThe optimistic concurrency control that reads multiple shards cannot use simple CAS. It probably needs to do something like two-phase committing