4 ms·
less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still
by userbinator 1y ago
less than 60ns loopback latency
There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.
- mrlongroots 1y agoThe other funny bit is that one-way PCIe latency is 250ns-ish (don't quote me on the exact numbers), which imposes a hard 1us constraint on latency between two hosts.
- mgrosvenor 1y agoYou can go quicker with CXL, but not by much.
- foobar10000 1y agoThere are always DPUs like NextSilicon, NVidia's Blue*, etc - at under 100 ns of SoF to fast compute though.
- rasz 1y agoCut Through switching needs what, 14 bytes? Thats ~12ns on 10G? How would you do it in 40 cycles?
- Perz1val 1y agoNot my are of expertise, but afaik the neat part is that you don't do it in cycles. The "program" lives as the transistor layout, not as instructions
- wtallis 1y agoThe cycles in question are not those of any processor, but those of the signal coming in the wire. No matter how it's implemented, a switch cannot react to bits that are still in the wire and haven't made it to the switch yet.
- ongy 1y agowhy do you assume the full ethernet header? IIRC. cut-through only needs the first 6 bytes. Since it only needs the destination address for the port lookup. Potentially the first bit, on broadcast.
- rasz 1y agoBecause you might want VLAN. Plus you cant just start blasting reply without a preamble, so its still 14 bytes after receiving just the dest MAC. Then you get potential IFG, FEC, scrambling, it all adds up, no way any switch can do 4ns without heavy lawyer talk in the small print.
- rrr_eady 1y agoFor switching, this is just not true. There's in/out solutions that are at this kind of scale, but they are by definition not implementing any switching logic.
- jacquesm 1y agoIt's a bit more complicated than that. Loopback implies that the data went off the wire and was sent back. That puts some serious constraints on the process that deals with that data, how it is stored and how it is fed back into the system. You can't just compare that with naked pass-through latency (which is what a switch would do), because the signal is already on the way out before it has even fully been received. Essentially a few ns after the header is received it will already be passed on to the output port after some signal conditioning, which happens almost without latency. A typical 'high speed' switch will have 800 ns round trip, so that 60 ns quoted here is actually quite impressive.
- darrin 1y agoThere are L1 crosspoint "switches" in the 2-5ns range depending on port density and similar modes in some Ethernet silicon. These are not Ethernet switches in any normal sense though. They only replicate the signal 1:1 or 1:n and do not dynamically switch the destination based on anything in the packet. The fastest L3 Ethernet switches on the market are ~90ns.
- brewmarche 1y agoAre L1 switches like hubs or repeaters then? Sorry for my confusion.
- darrin 1y agoThey're more like digital patch panels. You can connect any pair of ports together in both directions (TX/RX), just like moving a patch cable around. You can also connect an incoming RX lane to multiple TX lanes (think optical splitter but electrical). You cannot merge signals together. Some products have FPGAs that add L2+ or multiplexing functionality with packet buffers, but that's not part of the 2-5ns path.
- brewmarche 1y agoSo, if I know the destination addresses on each port in advance I can program it to behave like an L2 switch (with proper multicast even), and I can use code to reprogram it whenever the traffic pattern changes?
- darrin 1y agoThe L1 path can't do any address matching and you can't overlap sources - like Ghostbusters, no crossing the streams. You can source port 1 to ports 2-4 (multicast, but static and unidirectional) and you can change a bidir connection between pairs of ports. The signal gets replicated regardless of what's in the frame. It wouldn't be practical to make it behave quite like an L2 switch.
- mgrosvenor 1y agoNo they don’t. At least not on the critical path. L1 switch’s at 4ns simply replicate the incoming electrical signal. That’s all they do. They also do some packet accounting and cute features off the critical path. None of that at 4ns.
- rrr_eady 1y agoThis guy knows a thing or two about these devices ;)
- foobar10000 1y agoYes he very much does :)
- foobar10000 1y agoIn fairness, assuming 4/8 ports in, 4 ports out, operating at some ungodly GHz using a custom GaAs or SiGe chip, and working in gearboxed scrambled space with very clever input mac prefix mapping and output scrambled/gearboxed precomputation, one _could_ do around 8ns optimistically from the start of the first 64b/66b block (the 0x55... preamble) for 10GB. There's some stuff about preamble shrink that makes it wonky, but that is doable. And interestingly enough, for 25G-R, one can comfortably do this in about 4ns. I am not in fact aware of such a beastie existing for 25G, but I have seen 1 or 2 for 10G though. Surprisingly, if ChatGPT is prompted _juust_ right, it will even give you a good way to do this.
- throw0101d 1y ago> There are some Ethernet switches with 4ns latency […] E.g., * https://www.arista.com/en/products/7130-connect https://www.arista.com/en/products/7130-connect * https://www.arista.com/en/products/7130-series https://www.arista.com/en/products/7130-series