4 ms·

Yup. Going back to the principles of distributed systems, I’m really wondering how “NTP but machine learning” can be used to minimize the clock skew to hundreds

Yup. Going back to the principles of distributed systems, I’m really wondering how “NTP but machine learning” can be used to minimize the clock skew to hundreds of nanoseconds with software only. The RTT between two servers can vary a lot — and way past the calculated minimum RTT. The figures (eg $21M) and names dropped (eg Stanford) are an appeal to authority, which does make me curious. I’d love to see some papers, I went through all of Balaji Prabhakar‘s publications (titles only), and didn’t see a single paper that sounded like “NTP but machine learning. If anyone else knows more about this, I’d love to hear from you. Surely $21M doesn’t get dropped with at least someone doing due dill on the tech?

kloading 5y ago
I also haven't seen any work of his using ML, that may just be a buzzword thrown in the PR release. But if anyone's interested, I believe this is the paper alluded to in the article: Clock synchronization: https://www.usenix.org/conference/nsdi18/presentation/geng https://www.usenix.org/conference/nsdi18/presentation/geng I think the TechCrunch article doesn't really explain their application of clock synchronization well. Here are the other relevant papers and my attempt at explaining the general idea below. Network edge-based measurement (one-way delay?): https://www.usenix.org/system/files/nsdi19-geng.pdf https://www.usenix.org/system/files/nsdi19-geng.pdf Congestion Control: https://www.usenix.org/system/files/nsdi21-liu.pdf https://www.usenix.org/system/files/nsdi21-liu.pdf Each paper I've listed builds on the last. Their method to synchronize clocks made accurate and efficient measurement of one-way delay possible with commodity hardware. This measurement of one-way delay occurs at the edge, allowing them to "hold" incoming packets at the edge for extremely small periods to reduce congestion (latency) while maintaining throughput. From my understanding, traditional congestion control algorithms require rich telemetry from the entire network, which is likely not accessible in a public cloud environment. Balaji and clockwork's algorithms only need to make these measurements from the edge (which customers in public cloud have access to). I'm curious to see how all this will scale for multi-region deployments. If the latency between VMs from region 1 and region 2 is significant, I wonder if the measurement will actually be useful in deciding to "hold" the packets.
- galeaspablo 5y ago
  This is exactly what I was looking for. Thank you. It’s a shame fundamental publications aren’t part of PR articles. I’d also love to see better details on their website. Oh well, I cant expect everything to work like academia — c’est la vie.
- shiftingleft 5y ago
  > I also haven't seen any work of his using ML, that may just be a buzzword thrown in the PR release. The first article you reference includes something about Support Vector Machines (SVMs).
- danzheng 5y ago
  I'm from the Clockwork team, thanks for listing the relevant papers. Accurate clocks sync enables true one-way delay measurements (instead of RTT/2), this allows for edge-based network visibility. We launched Latency Sensei beta – a sensor, monitor and auditor that provides visibility into cloud deployments. The gallery has cloud fitness reports on GCP, AWS and Azure. Some interesting reports include: 1)how VM colocation impairs network bandwidth, and 2) tale of 2 cloud regions, London vs Singapore. Take a look and we'd love to get some feedback https://sensei.clockwork.io/user/gallery/ https://sensei.clockwork.io/user/gallery/ On Congestion control, an edge-based solution is coming soon. If you're interested in a private beta, email us at hello@clockwork.io
isatty 5y ago
> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? You’d be surprised, I was.
insaneirish 5y ago
> The figures (eg $21M) and names dropped (eg Stanford) are an appeal to authority, which does make me curious. They have the 'names' because they are a reputable group of people. They have been at this problem for a while, first with extensive research that required clock sync as a pre-requisite, and then to evolving into clock sync as a formidable problem unto itself. Clockwork is a rename of the company as far as I can tell. Their original name (Tick Tock Networks) [1] was probably too close to what has become a very popular homophone. [1]: https://opencorporates.com/companies/us_de/6947372 https://opencorporates.com/companies/us_de/6947372
- galeaspablo 5y ago
  Sure, but ‘names’ don’t explain how they do what they claim to do. Another reply kindly provided references to publications (which I couldn’t find myself) — they made for great reading. You seem to know their work, so if you have further publications, I’d love to get them please.
  - insaneirish 5y ago
    Here's a talk Balaji Prabhakar gave: https://www.youtube.com/watch?v=YQSA99nrFEU https://www.youtube.com/watch?v=YQSA99nrFEU. Looks like the clock sync part starts about 30 minutes in. Haven't watched this one, but might be complementary: https://www.youtube.com/watch?v=Opf9CBwP5R8 https://www.youtube.com/watch?v=Opf9CBwP5R8.
unmole 5y ago
> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? Juicero raised $120M.
- charcircuit 5y ago
  Juicero only raised $18.5M in their series A.
  - unmole 5y ago
    And?
ahazred8ta 5y ago
From the article, time synchronization is only a small part of what they do. Their Big Thing is traffic shaping and latency management within and between datacenters.
- galeaspablo 5y ago
  Those products are built on time synchronization. E.g., a latency measurement needs a _true_ difference between the timestamps of two servers.
jacquesm 5y ago
> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? Just some words: Theranos. Ubeam. Moller. And many, many others. Investors do this all the time. FOMO + 'wouldn't it be nice' are powerful ways to part fools from their money.

kloading 5y ago

I also haven't seen any work of his using ML, that may just be a buzzword thrown in the PR release. But if anyone's interested, I believe this is the paper alluded to in the article: Clock synchronization: https://www.usenix.org/conference/nsdi18/presentation/geng https://www.usenix.org/conference/nsdi18/presentation/geng I think the TechCrunch article doesn't really explain their application of clock synchronization well. Here are the other relevant papers and my attempt at explaining the general idea below. Network edge-based measurement (one-way delay?): https://www.usenix.org/system/files/nsdi19-geng.pdf https://www.usenix.org/system/files/nsdi19-geng.pdf Congestion Control: https://www.usenix.org/system/files/nsdi21-liu.pdf https://www.usenix.org/system/files/nsdi21-liu.pdf Each paper I've listed builds on the last. Their method to synchronize clocks made accurate and efficient measurement of one-way delay possible with commodity hardware. This measurement of one-way delay occurs at the edge, allowing them to "hold" incoming packets at the edge for extremely small periods to reduce congestion (latency) while maintaining throughput. From my understanding, traditional congestion control algorithms require rich telemetry from the entire network, which is likely not accessible in a public cloud environment. Balaji and clockwork's algorithms only need to make these measurements from the edge (which customers in public cloud have access to). I'm curious to see how all this will scale for multi-region deployments. If the latency between VMs from region 1 and region 2 is significant, I wonder if the measurement will actually be useful in deciding to "hold" the packets.

galeaspablo 5y ago
This is exactly what I was looking for. Thank you. It’s a shame fundamental publications aren’t part of PR articles. I’d also love to see better details on their website. Oh well, I cant expect everything to work like academia — c’est la vie.
shiftingleft 5y ago
> I also haven't seen any work of his using ML, that may just be a buzzword thrown in the PR release. The first article you reference includes something about Support Vector Machines (SVMs).
danzheng 5y ago
I'm from the Clockwork team, thanks for listing the relevant papers. Accurate clocks sync enables true one-way delay measurements (instead of RTT/2), this allows for edge-based network visibility. We launched Latency Sensei beta – a sensor, monitor and auditor that provides visibility into cloud deployments. The gallery has cloud fitness reports on GCP, AWS and Azure. Some interesting reports include: 1)how VM colocation impairs network bandwidth, and 2) tale of 2 cloud regions, London vs Singapore. Take a look and we'd love to get some feedback https://sensei.clockwork.io/user/gallery/ https://sensei.clockwork.io/user/gallery/ On Congestion control, an edge-based solution is coming soon. If you're interested in a private beta, email us at hello@clockwork.io

isatty 5y ago

> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? You’d be surprised, I was.

insaneirish 5y ago

> The figures (eg $21M) and names dropped (eg Stanford) are an appeal to authority, which does make me curious. They have the 'names' because they are a reputable group of people. They have been at this problem for a while, first with extensive research that required clock sync as a pre-requisite, and then to evolving into clock sync as a formidable problem unto itself. Clockwork is a rename of the company as far as I can tell. Their original name (Tick Tock Networks) [1] was probably too close to what has become a very popular homophone. [1]: https://opencorporates.com/companies/us_de/6947372 https://opencorporates.com/companies/us_de/6947372

galeaspablo 5y ago
Sure, but ‘names’ don’t explain how they do what they claim to do. Another reply kindly provided references to publications (which I couldn’t find myself) — they made for great reading. You seem to know their work, so if you have further publications, I’d love to get them please.
- insaneirish 5y ago
  Here's a talk Balaji Prabhakar gave: https://www.youtube.com/watch?v=YQSA99nrFEU https://www.youtube.com/watch?v=YQSA99nrFEU. Looks like the clock sync part starts about 30 minutes in. Haven't watched this one, but might be complementary: https://www.youtube.com/watch?v=Opf9CBwP5R8 https://www.youtube.com/watch?v=Opf9CBwP5R8.

unmole 5y ago

> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? Juicero raised $120M.

charcircuit 5y ago
Juicero only raised $18.5M in their series A.
- unmole 5y ago
  And?

ahazred8ta 5y ago

From the article, time synchronization is only a small part of what they do. Their Big Thing is traffic shaping and latency management within and between datacenters.

galeaspablo 5y ago
Those products are built on time synchronization. E.g., a latency measurement needs a _true_ difference between the timestamps of two servers.

jacquesm 5y ago

> Surely $21M doesn’t get dropped with at least someone doing due dill on the tech? Just some words: Theranos. Ubeam. Moller. And many, many others. Investors do this all the time. FOMO + 'wouldn't it be nice' are powerful ways to part fools from their money.