5 ms·
HFT guy here. There's problems with using the internet if you want low latency. Not just low latency, but also low jitter, ie consistent latency. You're sharin
by lordnacho 8y ago
HFT guy here.
There's problems with using the internet if you want low latency. Not just low latency, but also low jitter, ie consistent latency. You're sharing a bunch of infrastructure with other people, so sometimes your packet will have to wait. Try to ping some internet server a few times and you'll see it changes. I've done this with rented lines, and the number is the same every time.
Not inherent to the internet is the fact that REST seems to be a standard. Probably people will want a REST interface. So that means serializing and deserializing messages, and the messages are always going to be a lot longer than they need to be. It's not that you have to do it this way, though. There's also the fact that internet architectures tend to have a bunch of hops (load balancer / gateway / etc), which isn't going to help, but definitely something you could work around.
The figures he gives are interesting. If you really need under 20ms for AR to work, how is remote surgery going to work? 20ms is close to the time you get to render a frame in real time games at 60fps, seems a bit fast to me. The surgeon can't be far away if this is true.
- ignoramous 8y agoThanks. Want to point to others that might be looking at the parent comment and wondering 'what on earth...' A talk on ultra low latency, shared recently, was a great resource and expands on and adds to some of the points made above: https://news.ycombinator.com/item?id=18729822 https://news.ycombinator.com/item?id=18729822
- devereaux 8y agoWhen you start needing ultra low latency, you see many things very differently. Take timestamping for example. NTP has unacceptable jitter.
- devereaux 8y agoRegarding hardware: I try to not share infrastructure. What I can not fully control can and will dysfunction, even in datacenters. For example, the LAN options are most often VLAN. So for my servers, I have extra NICs and direct connections one-to-one by crossover cable and ARP monitoring. It costs more, but it quickly pays for itself. For software, everything is done in house. As you said, it doesn't have to be done a way or another.
- walrus01 8y agoI was under the impression that the most serious HFT doesn't even use Ethernet anymore. At each end, where it meets a common buy/sell platform, yes. But not for the competing HFT data link companies' inter-city messaging. For the shortwave links in place between NY-Tokyo, for several I'm privy to details of, people have written their own custom dedicated purpose protocols that implement very low bitrate, ultra low latency data messaging with less overhead and ASIC packet processing time.
- devereaux 8y agoDepends on what you do. For what I do, Ethernet is acceptable as long as the latency is below 1 ms. Here're some quick stats for what I consider good enough this specific usecase: rtt min/avg/max/std-dev = 0.069/0.095/0.121/0.016 ms Generally it is better to keep things simple and use existing stuff whenever possible. If we are talking about very low latency (say below 30 us) you may need many other things first, like a good timesource. Then indeed, when you start rolling your own stratum1 NTP servers, rolling your own protocols can be a sensible answer to the problem you have.
- MuffinFlavored 8y agoAt a high level, what do you do with HFT? You aren't doing the trading, right? You just manage the systems behind it? So what is? Are there really bots "in production" that are making buying and selling decisions tied to real money so quickly that <1ms is needed?
- walrus01 8y agoI'm not directly involved with it, but tangentially at a network engineering level. In things like the NYC-Chicago path, 1 millisecond is an ETERNITY. The actual people working for HFT firms are probably not going to provide much insight into what's going on behind the scenes, due to very restrictive NDAs, security policies and litigious corporate overlords.
- 8y ago
- bewo001 8y agoRemote surgery is one of those 5G hype applications that nobody will really offer. What liability insurance will cover if a fiber gets cut and the resulting fail-over delay and jitter causes the robot arm to nick an important artery or nerve? For AR/VR, you need to detect the motion, calculate the scenery, and put it on screen. This alone eats up a big part of the 20 ms budget. And to avoid motion sickness, the delay should be even shorter. First Person View / FPV quadcopter pilots still use analog TV transmission as it has a latency of about 26 ms. As the 'dots' are sent as they are sampled by the camera, expert pilots can judge from distortions in a picture how the quadcopter has moved between two full frames. When I looked it up, the best digital FPV video systems offered about 50/60 ms delay.