4 ms·

Oh joys, the sound of re-inventing the wheel. The old "not invented here" syndrome. Per the blog, they state they want max 7ms difference between nodes. I'm n

Oh joys, the sound of re-inventing the wheel. The old "not invented here" syndrome. Per the blog, they state they want max 7ms difference between nodes. I'm no timesync guru, but AFAIK can do that off-the-shelf today using chrony with hardware timestamping or PTP. No need to invent your own. And if you still insist on doing something different, why not work on improving the existing work being done under the the Open Compute Time Appliance Project (https://www.opencompute.org/projects/time-appliances-project-tap https://www.opencompute.org/projects/time-appliances-project...).

rkarthik007 5y ago
> but AFAIK can do that off-the-shelf today using chrony with hardware timestamping or PTP. No need to invent your own. Actually, the issue is about the max clock skew guarantees (as opposed to the average or median). Even a single violation of this breaks the ACID semantics. So, we do use chrony, but need all this to ensure there is a max guarantee. We would totally have adopted an existing solution - we did look at all alternatives available. Disclosure: one of the founders of the YugabyteDB project
- traceroute66 5y ago
  How would you say your solution compares to the CockroachDB solution[1] ? [1] https://www.cockroachlabs.com/docs/stable/architecture/transaction-layer.html#time-and-hybrid-logical-clocks https://www.cockroachlabs.com/docs/stable/architecture/trans...
  - rkarthik007 5y ago
    This blog post covers the various techniques to sync clocks in general (NTP, GPS clocks / TrueTime, Timestamp Oracle, HLC, etc). CockroachDB uses HLC (hybrid logical clocks), which is the same clock sync mechanism as YugabyteDB uses (and Apache Kudu as well). The page you linked to talks about how HLC is used to get distributed transactions to work, and there are differences here between the different databases (Spanner, CRDB, YugabyteDB, etc).
jeffbee 5y ago
Good luck using PTP in EC2.
- yangosoft 5y ago
  Aren't they providing NIC with hardware timestamp? That's bad.
  - jeffbee 5y ago
    ENA does not have this capability, or the capability is not exposed to the instance. Perhaps some of the older instance types with different network adapters have it, but I never tried them.
    - yangosoft 5y ago
      Understood. Thank you!

rkarthik007 5y ago

> but AFAIK can do that off-the-shelf today using chrony with hardware timestamping or PTP. No need to invent your own. Actually, the issue is about the max clock skew guarantees (as opposed to the average or median). Even a single violation of this breaks the ACID semantics. So, we do use chrony, but need all this to ensure there is a max guarantee. We would totally have adopted an existing solution - we did look at all alternatives available. Disclosure: one of the founders of the YugabyteDB project

traceroute66 5y ago
How would you say your solution compares to the CockroachDB solution[1] ? [1] https://www.cockroachlabs.com/docs/stable/architecture/transaction-layer.html#time-and-hybrid-logical-clocks https://www.cockroachlabs.com/docs/stable/architecture/trans...
- rkarthik007 5y ago
  This blog post covers the various techniques to sync clocks in general (NTP, GPS clocks / TrueTime, Timestamp Oracle, HLC, etc). CockroachDB uses HLC (hybrid logical clocks), which is the same clock sync mechanism as YugabyteDB uses (and Apache Kudu as well). The page you linked to talks about how HLC is used to get distributed transactions to work, and there are differences here between the different databases (Spanner, CRDB, YugabyteDB, etc).

jeffbee 5y ago

Good luck using PTP in EC2.

yangosoft 5y ago
Aren't they providing NIC with hardware timestamp? That's bad.
- jeffbee 5y ago
  ENA does not have this capability, or the capability is not exposed to the instance. Perhaps some of the older instance types with different network adapters have it, but I never tried them.
  - yangosoft 5y ago
    Understood. Thank you!