6 ms·

Why not get rid of NTP altogether? For servers, they ought to be able to accept a hardware time and frequency inputs. Typically this would mean a 1 Pulse-Per-

Why not get rid of NTP altogether? For servers, they ought to be able to accept a hardware time and frequency inputs. Typically this would mean a 1 Pulse-Per-Second (PPS) signal and a 10 MHz signal. You might also want some kind of serial signal that describes the date and time to label the last second pulse. Why don't any professional servers come with this kind of connection built-in? Why can't I synchronize my Xeon chip's clock to my own reference frequency?

xenophonf 10y ago
1-second-per-second is a difficult engineering challenge: https://rachelbythebay.com/w/2014/06/14/time/ https://rachelbythebay.com/w/2014/06/14/time/ (associated HN discussion - https://news.ycombinator.com/item?id=8066915 https://news.ycombinator.com/item?id=8066915) Here's a choice quote from the NTP FAQ (http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm): Unfortunately all the common clock hardware is not very accurate. This is simply because the frequency that makes time increase is never exactly right. Even an error of only 0.001% would make a clock be off by almost one second per day. This is also a reason why discussing clock problems uses very fine measures: One PPM (Part Per Million) is 0.0001% (1E-6). And if you want to delve into the physics, you'll enjoy this tutorial by John Vig (U.S. Army Communications-Electronics Command): http://www.am1.us/Local_Papers/U11625%20VIG-TUTORIAL.pdf http://www.am1.us/Local_Papers/U11625%20VIG-TUTORIAL.pdf
- someguydave 10y ago
  That might have been true awhile ago, but today you can get GPS-disciplined oscillators that can produce 1 PPS accurate down to tens of nanoseconds. By accurate, I mean "replicates what the NIST or USNO atomic clocks indicate."
- someguydave 10y ago
  Also your "rachelbythebay" post is discussing exactly the problem I'm trying to solve with hardware - using software. You don't need to run NTPd at all if your computer hardware and infrastructure has solved the problem of "keep my clock accurate down to 10's nanoseconds." Also, there's another unrelated problem which is "how accurately does a user process running in a linux kernel get the current time?". That's a problem no matter what kind of hardware the kernel is running on (although hardware might assist with that as well).
walrus01 10y ago
> For servers, they ought to be able to accept a hardware time and frequency inputs. Typically this would mean a 1 Pulse-Per-Second (PPS) signal and a 10 MHz signal. You might also want some kind of serial signal that describes the date and time to label the last second pulse. Because large numbers of servers in a mass-scale deployment frequently have only two things: Ethernet connections and power. Introducing coaxial cable so you can feed a 10MHz reference signal (such as you feed a reference clock signal to certain RF Rx and Tx equipment) is a non starter. Introducing a serial cable to every server into an environment that is purely IP/Ethernet is a non starter.
- someguydave 10y ago
  So people would be willing to spend millions of dollars on computers and would also be unwilling to spend a few thousand more for cabling?
  - walrus01 10y ago
    You underestimate the complication of cabling and managing that cabling when building a facility that is on the scale of eight thousand or more 1U size servers (or the proprietary open compute/blade type equivalent of the same in 1U). Then the RF and/or serial cabling from each server has to go somewhere, to some equipment (what, a new special 10 MHz reference 1U top of rack device that is a bespoke creation?), which both consumes rack space and electricity. And is managed over some sort of management network.
    - someguydave 10y ago
      >what, a new special 10 MHz reference 1U top of rack device that is a bespoke creation? It's not a "bespoke creation", such things have been available for decades: https://www.microsemi.com/products/timing-synchronization-systems/time-frequency-distribution/gps-instruments/xli#overview https://www.microsemi.com/products/timing-synchronization-sy...
      - walrus01 10y ago
geofft 10y ago
1. If I have a fleet of servers, it's a lot more important to maintain accurate time between the servers than accurate time in an absolute sense. If your hardware time cable can't span two racks, I don't want two separate clocks on each rack from a different hardware source. 2. For applications where you need extremely highly accurate time, there already are other solutions like PTP. NTP is mainly useful for wide-area connections. I want to keep my laptop's time accurate, so that if I modify a file here and rsync it elsewhere, make doesn't get confused about the time my laptop sent. I'm not plugging in a hardware time server to my laptop.
- someguydave 10y ago
  PTP is more accurate than NTP, but not as accurate as a 1PPS line. However, PTP has the advantage that is bidirectional so it can solve the (non-trival) problem of correcting for propagation delays.
- someguydave 10y ago
  >If your hardware time cable can't span two racks, I don't want two separate clocks on each rack from a different hardware source. It absolutely would span racks. You must account for the additional speed-of-light delay the cable itself would add though.
mcguire 10y ago
I wonder how that would work with virtual servers?
- someguydave 10y ago
  Does the hardware clock that runs the physical chip also provide a clock for the underlying virtual cpu?
jlgaddis 10y ago
You can. The workstation in front of me is directly connected (via serial) to a GPS providing a 1PPS signal. Everything else in my house uses NTP and, in some cases, PTP, however.
- someguydave 10y ago
  Yes but the oscillator in your computer is almost certainly not referenced to a frequency reference. Also you are not able to correct that oscillator's frequency with your hardware (yes, you can estimate your frequency error from external PTP measurements but you cannot physically correct it - nor prevent drift).

xenophonf 10y ago

1-second-per-second is a difficult engineering challenge: https://rachelbythebay.com/w/2014/06/14/time/ https://rachelbythebay.com/w/2014/06/14/time/ (associated HN discussion - https://news.ycombinator.com/item?id=8066915 https://news.ycombinator.com/item?id=8066915) Here's a choice quote from the NTP FAQ (http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm): Unfortunately all the common clock hardware is not very accurate. This is simply because the frequency that makes time increase is never exactly right. Even an error of only 0.001% would make a clock be off by almost one second per day. This is also a reason why discussing clock problems uses very fine measures: One PPM (Part Per Million) is 0.0001% (1E-6). And if you want to delve into the physics, you'll enjoy this tutorial by John Vig (U.S. Army Communications-Electronics Command): http://www.am1.us/Local_Papers/U11625%20VIG-TUTORIAL.pdf http://www.am1.us/Local_Papers/U11625%20VIG-TUTORIAL.pdf

someguydave 10y ago
That might have been true awhile ago, but today you can get GPS-disciplined oscillators that can produce 1 PPS accurate down to tens of nanoseconds. By accurate, I mean "replicates what the NIST or USNO atomic clocks indicate."
someguydave 10y ago
Also your "rachelbythebay" post is discussing exactly the problem I'm trying to solve with hardware - using software. You don't need to run NTPd at all if your computer hardware and infrastructure has solved the problem of "keep my clock accurate down to 10's nanoseconds." Also, there's another unrelated problem which is "how accurately does a user process running in a linux kernel get the current time?". That's a problem no matter what kind of hardware the kernel is running on (although hardware might assist with that as well).

walrus01 10y ago

> For servers, they ought to be able to accept a hardware time and frequency inputs. Typically this would mean a 1 Pulse-Per-Second (PPS) signal and a 10 MHz signal. You might also want some kind of serial signal that describes the date and time to label the last second pulse. Because large numbers of servers in a mass-scale deployment frequently have only two things: Ethernet connections and power. Introducing coaxial cable so you can feed a 10MHz reference signal (such as you feed a reference clock signal to certain RF Rx and Tx equipment) is a non starter. Introducing a serial cable to every server into an environment that is purely IP/Ethernet is a non starter.

someguydave 10y ago
So people would be willing to spend millions of dollars on computers and would also be unwilling to spend a few thousand more for cabling?
- walrus01 10y ago
  You underestimate the complication of cabling and managing that cabling when building a facility that is on the scale of eight thousand or more 1U size servers (or the proprietary open compute/blade type equivalent of the same in 1U). Then the RF and/or serial cabling from each server has to go somewhere, to some equipment (what, a new special 10 MHz reference 1U top of rack device that is a bespoke creation?), which both consumes rack space and electricity. And is managed over some sort of management network.
  - someguydave 10y ago
    >what, a new special 10 MHz reference 1U top of rack device that is a bespoke creation? It's not a "bespoke creation", such things have been available for decades: https://www.microsemi.com/products/timing-synchronization-systems/time-frequency-distribution/gps-instruments/xli#overview https://www.microsemi.com/products/timing-synchronization-sy...
    - walrus01 10y ago

geofft 10y ago

1. If I have a fleet of servers, it's a lot more important to maintain accurate time between the servers than accurate time in an absolute sense. If your hardware time cable can't span two racks, I don't want two separate clocks on each rack from a different hardware source. 2. For applications where you need extremely highly accurate time, there already are other solutions like PTP. NTP is mainly useful for wide-area connections. I want to keep my laptop's time accurate, so that if I modify a file here and rsync it elsewhere, make doesn't get confused about the time my laptop sent. I'm not plugging in a hardware time server to my laptop.

someguydave 10y ago
PTP is more accurate than NTP, but not as accurate as a 1PPS line. However, PTP has the advantage that is bidirectional so it can solve the (non-trival) problem of correcting for propagation delays.
someguydave 10y ago
>If your hardware time cable can't span two racks, I don't want two separate clocks on each rack from a different hardware source. It absolutely would span racks. You must account for the additional speed-of-light delay the cable itself would add though.

mcguire 10y ago

I wonder how that would work with virtual servers?

someguydave 10y ago
Does the hardware clock that runs the physical chip also provide a clock for the underlying virtual cpu?

jlgaddis 10y ago

You can. The workstation in front of me is directly connected (via serial) to a GPS providing a 1PPS signal. Everything else in my house uses NTP and, in some cases, PTP, however.

someguydave 10y ago
Yes but the oscillator in your computer is almost certainly not referenced to a frequency reference. Also you are not able to correct that oscillator's frequency with your hardware (yes, you can estimate your frequency error from external PTP measurements but you cannot physically correct it - nor prevent drift).