3 ms·

> I have been working on algorithmic trading and it is not that hard to reliably (like 100% of the time) respond within couple microseconds. As someone who wor

> I have been working on algorithmic trading and it is not that hard to reliably (like 100% of the time) respond within couple microseconds. As someone who works in the flight test instrumentation industry (that primarily uses PTP rather than NTP), my first thought was "is a couple of microseconds supposed to be good?". With PTP, I usually achieve RMS offsets of < 50 nanoseconds over gigabit Ethernet. I sort of agree with your point in that it's probably pointless implementing NTP inside of an FPGA, but I'd like to extend it further and say that if you really care about accurate timing, you should be using PTP rather than NTP anyway. In that case, an FPGA solution makes much more sense. However, in the systems I've worked with, the FPGA only implements a few pieces of core functionality - like the frame time stamping and the numerically-frequency-controlled clock. The actual protocol is usually implemented in software, except in the case of transparent switches. That said, most modern NICs have PTP hardware, but you still usually need some external logic to actually use these synchronized clocks.

lmilcin 6y ago
On a "normal" box the time is usually combination of various sources neither of which was really designed to provide accurate, sub microsecond, absolute time. (That except possibly for this "PTP hardware" in NICs that I don't know anything about) So even if you somehow devise a piece of hardware to pass the accurate time to the operating system, there would be no way to track it accurately. Now, I think that really accurate absolute timing on machines isn't really that critical. For example, for consistency protocols usually it is enough to provide guarantee of 1s around true time. For algorithmic trading we were mostly interested in responding as quickly as possible to the event that came from the stock exchange. The packets always came with a known delay and the timing wasn't all that important except for debugging. Everything was done on a single box (so no need to synchronize with anything else) and even on that box the events were processed such that there was no need to coordinate with other threads, each thread just consumed, processed and published results.
- mlyle 6y ago
  Most Intel 10 gigabit adapters support PTP. It's just about inserting the exact time the frame will be sent into a sent frame, and noting the exact time the frame is received. All that's left is a small amount of clock jitter and cable delays. High end switches support PTP themselves so you don't need to worry about queueing delays, but you don't need a switch that supports PTP to get high resolution, low jitter, low uncertainty time-- at least on reasonably sized networks that are usually not saturated. There are many algorithmic trading applications that rely upon high quality time derived from a single coherent source, along with telecommunications, control systems, instrumentation, etc. And there are other distributed systems approaches where true ordering is nice: yes, you can provide ordering within a system of dependent events using a Lamport clock, vector clock, etc, but without high quality time you can also correctly sequence causally related events originating outside your tightly coupled system in many use cases. That said: PTP relies on a tightly coupled master and realistically is intended for a tightly coupled, hierarchal system. NTP is a better "internet" time protocol-- lots of logic for clock precedence, averaging of multiple sources, slower control system, etc.
  - eqvinox 6y ago
    A lot of intel 1G adapters do too, likely including the one in your laptop. $ lspci -s 0:1f.6 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (4) I219-V (rev 21) $ sudo ethtool -T enp0s31f6 Time stamping parameters for enp0s31f6: Capabilities: hardware-transmit software-transmit hardware-receive software-receive software-system-clock hardware-raw-clock PTP Hardware Clock: 0 Hardware Transmit Timestamp Modes: off on Hardware Receive Filter Modes: none all ptpv1-l4-sync ptpv1-l4-delay-req ptpv2-l4-sync ptpv2-l4-delay-req ptpv2-l2-sync ptpv2-l2-delay-req ptpv2-event ptpv2-sync ptpv2-delay-req
    - mlyle 6y ago
      Yup-- I didn't know the prevalence so I didn't say. You're wrong though-- there's no ethernet adapter in my laptop ;)
      - 6y ago

lmilcin 6y ago

On a "normal" box the time is usually combination of various sources neither of which was really designed to provide accurate, sub microsecond, absolute time. (That except possibly for this "PTP hardware" in NICs that I don't know anything about) So even if you somehow devise a piece of hardware to pass the accurate time to the operating system, there would be no way to track it accurately. Now, I think that really accurate absolute timing on machines isn't really that critical. For example, for consistency protocols usually it is enough to provide guarantee of 1s around true time. For algorithmic trading we were mostly interested in responding as quickly as possible to the event that came from the stock exchange. The packets always came with a known delay and the timing wasn't all that important except for debugging. Everything was done on a single box (so no need to synchronize with anything else) and even on that box the events were processed such that there was no need to coordinate with other threads, each thread just consumed, processed and published results.

mlyle 6y ago
Most Intel 10 gigabit adapters support PTP. It's just about inserting the exact time the frame will be sent into a sent frame, and noting the exact time the frame is received. All that's left is a small amount of clock jitter and cable delays. High end switches support PTP themselves so you don't need to worry about queueing delays, but you don't need a switch that supports PTP to get high resolution, low jitter, low uncertainty time-- at least on reasonably sized networks that are usually not saturated. There are many algorithmic trading applications that rely upon high quality time derived from a single coherent source, along with telecommunications, control systems, instrumentation, etc. And there are other distributed systems approaches where true ordering is nice: yes, you can provide ordering within a system of dependent events using a Lamport clock, vector clock, etc, but without high quality time you can also correctly sequence causally related events originating outside your tightly coupled system in many use cases. That said: PTP relies on a tightly coupled master and realistically is intended for a tightly coupled, hierarchal system. NTP is a better "internet" time protocol-- lots of logic for clock precedence, averaging of multiple sources, slower control system, etc.
- eqvinox 6y ago
  A lot of intel 1G adapters do too, likely including the one in your laptop. $ lspci -s 0:1f.6 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (4) I219-V (rev 21) $ sudo ethtool -T enp0s31f6 Time stamping parameters for enp0s31f6: Capabilities: hardware-transmit software-transmit hardware-receive software-receive software-system-clock hardware-raw-clock PTP Hardware Clock: 0 Hardware Transmit Timestamp Modes: off on Hardware Receive Filter Modes: none all ptpv1-l4-sync ptpv1-l4-delay-req ptpv2-l4-sync ptpv2-l4-delay-req ptpv2-l2-sync ptpv2-l2-delay-req ptpv2-event ptpv2-sync ptpv2-delay-req
  - mlyle 6y ago
    Yup-- I didn't know the prevalence so I didn't say. You're wrong though-- there's no ethernet adapter in my laptop ;)
    - 6y ago