3 ms·
This design can be extended to support interactive transactions. Here's how: 1. Assign each transaction a log position. For strict serializability instead of
by ryanworl 7y ago
This design can be extended to support interactive transactions.
Here's how:
1. Assign each transaction a log position. For strict serializability instead of just serializability, make this the highest log position ever acknowledged successfully to a client. This can be batched for throughput.
2. Have each client record their read set and write set, which includes the objects/ranges the transaction read and the writes / operations to be performed.
3. Have clients persist this R/W set into the log, or send directly to the lock server if it is also the master assigning log positions. Again, use batching for throughput.
4. Have your lock server either as a part of the master processing assigning log positions, or have it tail the log separately. The lock server will receive batches of transactions, take locks in the log's defined order, then commit another entry to the log with the commit/abort decision for each.
5. Respond to the client with the commit / abort decision.
To make this easier to program you'll probably want to include a read-your-writes cache on the client.
You can also scale out this log by having clients commit their operations to multiple logs. The only thing that needs to be serialized is the locking and commit / abort decision making. These other logs can be sharded by key range or just chosen randomly, as long as the commit / abort decision log includes which other log the data itself was committed to.
FoundationDB works roughly like this. The terminology is different (resolver instead of lock server, R/W set is conflict ranges, log position is version number, etc) but this is basically how it works.
- millrawr 7y agoPersisting executed transactions to a log with their read/write sets, and then determining if they actually commit also starts to sound like http://web.eecs.umich.edu/~michjc/eecs584/Papers/cidr11_hyder.pdf http://web.eecs.umich.edu/~michjc/eecs584/Papers/cidr11_hyde... to me
- ryanworl 7y agoYes, it is more similar to Hyder than FoundationDB if you persist the R/W sets to the log itself. FoundationDB gets around this by only keeping the lock table in memory. When any process in the transaction pipeline fails, the master (which holds the version number) advances the version far enough into the future that any in-progress transaction would fail because it is too old.