4 ms·

yeah synchrozing clocks across distributed systems is really hard without expensive hardware

audioheavy 3y ago
...but for distributed databases specifically, you can use a different algorithm like Calvin[0] or Fauna[1] that do not require external atomic clocks… but the CS point and the wealth of info in research papers (in distributed systems stuff) are solid ...but there is a lot of noise in those software papers, too - you are often disappointed by fine print, or have good curators/thought-leaders [2] - we all should share names ;) enjoying the discussion though - very timely if you ask me. -L, author of [1] below. [0] - The original Calvin paper - https://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf https://cs.yale.edu/homes/thomson/publications/calvin-sigmod... [1] - How Fauna implements a variation of Calvin - https://fauna.com/blog/inside-faunas-distributed-transaction-engine-dte#a-flexible-layered-system https://fauna.com/blog/inside-faunas-distributed-transaction... [2] - A great article about Calvin by Mohammad Roohitavaf - https://www.mydistributed.systems/2020/08/calvin.html?m=1#:~:text=Calvin%20is%20a%20transaction%20scheduling,that%20unlike%202PL%20is%20deterministic https://www.mydistributed.systems/2020/08/calvin.html?m=1#:~....
kmod 3y ago
I hear this from tech people, but hft people are happily humming along with highly-synchronized clocks (mifid ii requires clocks to be synchronized to 100us). I wouldn't say it's "easy" but apparently if you need it then you do it and it's not that bad.
- joshuamorton 3y ago
  > (mifid ii requires clocks to be synchronized to 100us) That only applies if the clocks are within 1ms of each other, so around 100 miles (or equivalently: within a single cloud region), and only came in to force in 2014. The bound that Spanner-likes keep is ~3ms for datacenters across continents, and that was in 2012.

...but for distributed databases specifically, you can use a different algorithm like Calvin[0] or Fauna[1] that do not require external atomic clocks… but the CS point and the wealth of info in research papers (in distributed systems stuff) are solid ...but there is a lot of noise in those software papers, too - you are often disappointed by fine print, or have good curators/thought-leaders [2] - we all should share names ;) enjoying the discussion though - very timely if you ask me. -L, author of [1] below. [0] - The original Calvin paper - https://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf https://cs.yale.edu/homes/thomson/publications/calvin-sigmod... [1] - How Fauna implements a variation of Calvin - https://fauna.com/blog/inside-faunas-distributed-transaction-engine-dte#a-flexible-layered-system https://fauna.com/blog/inside-faunas-distributed-transaction... [2] - A great article about Calvin by Mohammad Roohitavaf - https://www.mydistributed.systems/2020/08/calvin.html?m=1#:~:text=Calvin%20is%20a%20transaction%20scheduling,that%20unlike%202PL%20is%20deterministic https://www.mydistributed.systems/2020/08/calvin.html?m=1#:~....

kmod 3y ago

I hear this from tech people, but hft people are happily humming along with highly-synchronized clocks (mifid ii requires clocks to be synchronized to 100us). I wouldn't say it's "easy" but apparently if you need it then you do it and it's not that bad.

joshuamorton 3y ago
> (mifid ii requires clocks to be synchronized to 100us) That only applies if the clocks are within 1ms of each other, so around 100 miles (or equivalently: within a single cloud region), and only came in to force in 2014. The bound that Spanner-likes keep is ~3ms for datacenters across continents, and that was in 2012.