7 ms·
Achieving super low latency for critical real world internet applications (2018)
- client4 8y agoI do low latency network architectue for HFTs; I'm not sure there's a real world application for shaving 50us off a latency for a route outside of this industry. That said I've been impressed with Cloudflares consistent global low latency while traveling. It's a habit to ping 1.1.1.1 and see if it goes above 15ms.
- altmind 8y agoIf you are looking for information on low-latency computing, including some real advice how to tune for lower latency, look for info on HFT. 5ms reaction time mentioned on these slides are far cry for what computers can do. trading servers reaction time packet-in packet-out even without fpga can take as little as 20 microseconds.
- 3chelon 8y agoYes, but they sit right on transatlantic fibre-optic trunk cables. Presumably leased lines? It's a totally different proposition to a public mobile internet.
- altmind 8y agoMany trading companies now utilize over-the-horizon radio. Its the fastest, most direct connection available. But even without this: - cpu thermal profiles and power governors - cpu affinity - non-evented io(if you prefer latency over the number of clients served) - network interrupt coalescing(actually, disabling coalescing), network msi-x, hardware offloads, 10gb fiber or DAC connections(very cheap nowadays) - lock-free and kinda-crdt data structures(between threads) - predictable memory allocations(better - no allocations on critical path) - specialized logging and tracing(no stdio) can drive you quite far.
- bloomer 8y agoWhat is over the horizon radio? I am familiar with over the horizon radar, but not radio for communications. Do you have any references?
- altmind 8y agoSorry, used wrong word. "Skywave" is a better term.
- walrus01 8y agoShortwave radio with big-ass yagis and dipoles, aimed at each other in point-to-point links. Very low data rates, carrying custom messaging protocols, but also measurably less latency on critical paths such as NY-Chicago or London-Tokyo. There is a big crossover in tech with shortwave band military over the horizon communications systems, and also serious ham radio enthusiasts who use big yagis and antenna rotators to talk to each other from opposite sides of the planet. Things like chained 6 and 11 GHz microwave between NY-Chicago, which was the "hot new thing" in HFT 7-8 years ago to achieve better than fiber latency on that route, seem quaint now.
- trishume 8y agoIf you want multi-token autocompletion that's really good and language agnostic check out TabNine: https://tabnine.com/ https://tabnine.com/ TabNine just works and is super fast, I love it.
- lordnacho 8y agoHFT guy here. There's problems with using the internet if you want low latency. Not just low latency, but also low jitter, ie consistent latency. You're sharing a bunch of infrastructure with other people, so sometimes your packet will have to wait. Try to ping some internet server a few times and you'll see it changes. I've done this with rented lines, and the number is the same every time. Not inherent to the internet is the fact that REST seems to be a standard. Probably people will want a REST interface. So that means serializing and deserializing messages, and the messages are always going to be a lot longer than they need to be. It's not that you have to do it this way, though. There's also the fact that internet architectures tend to have a bunch of hops (load balancer / gateway / etc), which isn't going to help, but definitely something you could work around. The figures he gives are interesting. If you really need under 20ms for AR to work, how is remote surgery going to work? 20ms is close to the time you get to render a frame in real time games at 60fps, seems a bit fast to me. The surgeon can't be far away if this is true.
- ignoramous 8y agoThanks. Want to point to others that might be looking at the parent comment and wondering 'what on earth...' A talk on ultra low latency, shared recently, was a great resource and expands on and adds to some of the points made above: https://news.ycombinator.com/item?id=18729822 https://news.ycombinator.com/item?id=18729822
- devereaux 8y agoWhen you start needing ultra low latency, you see many things very differently. Take timestamping for example. NTP has unacceptable jitter.
- devereaux 8y agoRegarding hardware: I try to not share infrastructure. What I can not fully control can and will dysfunction, even in datacenters. For example, the LAN options are most often VLAN. So for my servers, I have extra NICs and direct connections one-to-one by crossover cable and ARP monitoring. It costs more, but it quickly pays for itself. For software, everything is done in house. As you said, it doesn't have to be done a way or another.
- iabacu 8y agoThen there are mixed reality applications. It won’t need HFT levels of low latency, but will demand a lot more bandwidth. Many of the tricks won’t work there. That’ll require exploiting edge distribution to its limits, with an entire new level of internet infrastructure that doesn’t exist yet.
- gameswithgo 8y agoNetwork Next is a company working on this, not just for critical applications but also games: https://networknext.com/concept/ https://networknext.com/concept/
- acranox 8y agoIt's terrifying to hear him mention self-driving cars as a use case for super low latencies. If a few milliseconds of network latency compromises some safety system in a car, that's bad. I'd be engineering a car for worst case scenarios, not for reliance on a fast network. It makes me think of this. https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...
- howard941 8y agoIt's not clear why some folks are dredging the self-driving cars, vehicle-to-vehicle, and vehicle-to-infrastructure use cases out for 5G but when the rubber hits the road I suspect vehicles will deploy with dedicated short range communications for the bits that must be low latency and 5G for ordinary cellphone use. The wiki page has some starting places for DSRC info. https://en.wikipedia.org/wiki/Dedicated_short-range_communications https://en.wikipedia.org/wiki/Dedicated_short-range_communic... . It is terrifying.
- aphextron 8y agoIt turns out car-to-car local networking is not really necessary. The use of 4G/5G networking + simulation based modeling and prediction gives great results. That’s what companies like Nexar [0] are doing. [0] https://www.getnexar.com https://www.getnexar.com
- aaaaaaaaaaab 8y agoIf it requires network connectivity then by definition it’s not self-driving.
- aaaaaaaaaab 8y agoIf it needs network connectivity then by definition it’s not self-driving.
- smartmic 8y agoCould RAMCloud be a candidate for this … at least to tackle the datacenter level, and —isn't the rest more or less out of control? https://ramcloud.stanford.edu/docs/doxygen/md_README.html https://ramcloud.stanford.edu/docs/doxygen/md_README.html
- jayd16 8y agoFor VR and AR you really don't need these low latencies over the internet. As long as the visual output matches the users head movement the game assets can have noticable lag. Like most existing games, rubberbanding is an annoyance but not disorienting. For those use cases it would be better to come up with a full 3d (not just stereoscopic) streaming video format that allows for some local camera movement.
- bewo001 8y agoIn 5G circles there is this "vision" of VR/AR servers that reside in "edge datacenters" close to the customer. The user's movements are streamed to such a server which returns the calculated video stream. This doesn't make sense at all, of course. But you can put buzzwords like VR/AR on your shiny 5G slides..
- bsder 8y agoA whole talk about low latency networking and not a single mention about TSN (time sensitive networking)/AVB (audio video bridging) which actually does low-latency?