4 ms·

In a globally distributed database, the main problem is time control because different locations have different clocks. For example, assume that one table of th

by d__k 9y ago

In a globally distributed database, the main problem is time control because different locations have different clocks. For example, assume that one table of the database is located on Earth while another table is stored on Mars (4-13 minutes delay). Now we want to commit a transaction with these two tables from the moon while the result will be queried from Venus. I could not understand from this post if Fauna is able to correctly work under such conditions and whether it was designed for such use cases.

sleepydog 9y ago
"Globally distributed" should imply that it's confined to one planet.
- pkolaczk 9y ago
  Even then, the roundtrip time between continents can be in range of several hundreds milliseconds, which is far too much for many real time transactional systems.
- mangatmodi 9y ago
  @sleepydog, that was an example. Even in one planet, RTT in internet can be quite severe for realtime applications.
- vidarh 9y ago
  More generally, the point is that if you have two locations where the fastest you can send a unit of data is t, then the shortest possible time interval you can pass a transaction from one location to another is t. That puts a upper (EDIT: corrected from lower) bound on throughput of many types of operations, such as e.g. if you need to guarantee strict ordering of transactions (EDIT: in the worst case, where transactions are being issued from multiple nodes; best case you can optimize in a variety of ways). So while we won't have to deal with interplanetary latencies, it still matters greatly for throughput and consistency guarantees.
  - mattb314 9y ago
    Even assuming you meant "upper bound" instead of "lower bound", this isn't strictly true in theory. To achieve high throughput with large latency between nodes, all you have to do is group transactions into large batches and reach consensus on how they should be ordered (for example, you can run paxos to decide what a batch consists of, and then use any method you like to order within the batch). Running paxos at interplanetary latencies would obviously take some time, but since you can have multiple rounds in progress at once (or increase the size of each batch to be much larger than the latency), throughput isn't limited by latency. In practice, most people care a lot about latency, and achieving the desired latency puts a limit on throughput. See the Calvin DB paper for a more thorough explanation.
    - vidarh 9y ago
      Yes, upper bound. Lower bound on time it takes for the transaction to be globally available. You're right that there are operations that you can increase throughput of, and I was imprecise in that you are right you can order them "after the fact", but that's not very interesting. The scenario I had in mind was where clients talking to different nodes are issuing transactions that depend on the same items of data. In that case you either need to obtain a lock, in which case you best case need to wait for the latency interval (plus a margin or largest possible clock skew) to see if the other side wants a lock on the same object, or you need to be optimistically firing off transactions, but the best case then is that you just get your transaction in right before the remote side start operating on it, and they happen to just fire off another transaction, and so on. In reality there'd be slowdowns. If you're dealing with uncontended objects, you can do much better on average, but you can't guarantee better than the latency between nodes. There are certainly tons of special cases where you can optimize.
nkristoffersen 9y ago
Much like the Google spanner database. Was at a presentation earlier this month where they went into more detail. Very interesting product! https://www.wired.com/2017/02/spanner-google-database-harnessed-time-now-open-everyone/ https://www.wired.com/2017/02/spanner-google-database-harnes...
proc0 9y ago
I think the moment the internet goes galactic, we will need to implement special relativity algorithms to properly sync machine states, unless quantum computation takes off and we get a quantum entagled network!

by d__k 9y ago

sleepydog 9y ago

"Globally distributed" should imply that it's confined to one planet.

pkolaczk 9y ago
Even then, the roundtrip time between continents can be in range of several hundreds milliseconds, which is far too much for many real time transactional systems.
mangatmodi 9y ago
@sleepydog, that was an example. Even in one planet, RTT in internet can be quite severe for realtime applications.
vidarh 9y ago
More generally, the point is that if you have two locations where the fastest you can send a unit of data is t, then the shortest possible time interval you can pass a transaction from one location to another is t. That puts a upper (EDIT: corrected from lower) bound on throughput of many types of operations, such as e.g. if you need to guarantee strict ordering of transactions (EDIT: in the worst case, where transactions are being issued from multiple nodes; best case you can optimize in a variety of ways). So while we won't have to deal with interplanetary latencies, it still matters greatly for throughput and consistency guarantees.
- mattb314 9y ago
  Even assuming you meant "upper bound" instead of "lower bound", this isn't strictly true in theory. To achieve high throughput with large latency between nodes, all you have to do is group transactions into large batches and reach consensus on how they should be ordered (for example, you can run paxos to decide what a batch consists of, and then use any method you like to order within the batch). Running paxos at interplanetary latencies would obviously take some time, but since you can have multiple rounds in progress at once (or increase the size of each batch to be much larger than the latency), throughput isn't limited by latency. In practice, most people care a lot about latency, and achieving the desired latency puts a limit on throughput. See the Calvin DB paper for a more thorough explanation.
  - vidarh 9y ago
    Yes, upper bound. Lower bound on time it takes for the transaction to be globally available. You're right that there are operations that you can increase throughput of, and I was imprecise in that you are right you can order them "after the fact", but that's not very interesting. The scenario I had in mind was where clients talking to different nodes are issuing transactions that depend on the same items of data. In that case you either need to obtain a lock, in which case you best case need to wait for the latency interval (plus a margin or largest possible clock skew) to see if the other side wants a lock on the same object, or you need to be optimistically firing off transactions, but the best case then is that you just get your transaction in right before the remote side start operating on it, and they happen to just fire off another transaction, and so on. In reality there'd be slowdowns. If you're dealing with uncontended objects, you can do much better on average, but you can't guarantee better than the latency between nodes. There are certainly tons of special cases where you can optimize.

nkristoffersen 9y ago

Much like the Google spanner database. Was at a presentation earlier this month where they went into more detail. Very interesting product! https://www.wired.com/2017/02/spanner-google-database-harnessed-time-now-open-everyone/ https://www.wired.com/2017/02/spanner-google-database-harnes...

proc0 9y ago

I think the moment the internet goes galactic, we will need to implement special relativity algorithms to properly sync machine states, unless quantum computation takes off and we get a quantum entagled network!