6 ms·

So Arista has a switch with an FPGA - 7124FX. The market for the HFT has crashed. When Arista shipped the 7124S the HFT guys where using Cisco 4900 at 4ms and t

So Arista has a switch with an FPGA - 7124FX. The market for the HFT has crashed. When Arista shipped the 7124S the HFT guys where using Cisco 4900 at 4ms and the 7124S was at 600ns. It was quite a change. The markets went crazy. The chip was called Bali by Fulcrum Microsystems - Intel bought them. The follow on chip Alta was very very very late. Arista did the 7124SX which got the latency down to 500ns. The Bali itself was 300ns but the PHY chips added the additional latency (in and out). This switch replaced the 7124S but there was not a mad rush to upgrade. Going from 4ms to 600ns was an order of magnitude but going from 600ns to 500ns, not so much. Cisco who got killed by Arista in this market spent ~$100M and came up with the Nexus 3548. It had "warp mode" that could do ~50ns. This mode however required lots of pre-planning and was fixed. Market data feeds are multicast. The handoff from the exchange was 1G or even low as 100mb in the Asian markets back then. If you added up every feed you could buy it was around ~3G. You would never do this on one link. The servers for looking at the data would join groups to get the feed. The servers used 10G NICs when the traffic load on them was only around ~100MB. Once again for latency. Serialization delay was the key here. The market order would go back up to the market on another path. The idea was the HFT guys would want to process the data faster then the rest of the people. The link down to them and the link up to order was the same for everyone. If you went into the NYSE colo the cable length to the router was the same if you where in the rack next to it or on the other side of the datacenter. Anyway back to switches. So Arista shipped the 7150 around the same time as Cisco shipped the 3548. It was around 350ns using the Alta chip. The reality was that this was low enough and the traders started to look for other places to tweak. Calling this the fastest switch in the world and then printing 5ns is misleading. It is 5ns as a Layer 1 patch panel. Not really what you want for market data which needs multicast, PIM, IGMP snooping, BPG, ACL, etc.. For the 110s is this multicast with all features enabled? Let me know if I missed a link. BTW, if you look back when the 7124S was released there were others that built a switch based on the Bali chip like BNT. An important point to note is that the chip is line rate multicast but that is not enough. Processing the joins/leaves and programing the chip is a function of the software and that depends on the quality of the code and the CPU system in the switch. Arista won because of this. Here is a link to a bake off from 2010. http://www.networkworld.com/article/2241525/virtualization/arista--blade-win-top-spot-in-data-center-switch-test.html http://www.networkworld.com/article/2241525/virtualization/a...

deadgrey19 12y ago
> Calling this the fastest switch in the world and then printing 5ns is misleading. The claim in the (original) story title is 110ns layer 2+ switching. I'm sitting in the summit right now, no details on what 2+ means, but, the target market is market data distribution (and this was discussed in the slides) so I would assume that they've thought about this. 5ns is additional functionality available in previous products (http://exablaze.com/exalink-50 http://exablaze.com/exalink-50) which is also in this product. From the sides, the idea is that they can set up multiple broadcast groups with the 5ns device to distribute market data, and then aggregate responses together with with 110ns (or in some cases 100ns) device, all in one box.
- myrandomcomment 12y ago
  So the exalink-50 number is as a Layer1 switch. So the comment "can setup multiple broadcast groups" sounds like the Cisco Nexus 3548 Warp Mode. The market did not take to that so not sure how this is going to change that. Like I said, I have not reviewed all the data yet. An important step for verification for all this is to put it in the hands of David Newman for a test, either paid via his private company Network Test, or publicly via Network World. A testing house like EANTC would also be great. As you can might have guessed from my post I know a bit about this world (I am not involved anymore). At this point it is more of a historical curiosity on my part more then anything (I still have a number of friends that run HFT networks at the banks). Given what I saw in testing and claims during the early HFT rush I always want to see 3rd party testing from a reputable testing house showing a real market data feature set being used. Here is a starting list: 1. BGP running 2. Mulitcast w/PIM 3. IGMP snooping 4. ACL configured 5. 1 port to max port of switch fanout. 6. 1% to 100% line rate on the feed. 7. Packet size ranges, both fixed and mixed. 8. Join/leave time. 9. Max groups joined.
  - deadgrey19 12y ago
    Agreed. Independent verification is definitely needed. Although I think that this is why they've announced this at STAC which does this (https://stacresearch.com/about https://stacresearch.com/about). I've moved over to academia now, but I came from tan HFT prop-trading background. We would have jumped at the chance to get one of these in our hands, but this was 3 years ago now. Can I ask why you'd want BGP running in a private LAN? This seems like an odd thing to do, especially in Colo environments.

deadgrey19 12y ago

> Calling this the fastest switch in the world and then printing 5ns is misleading. The claim in the (original) story title is 110ns layer 2+ switching. I'm sitting in the summit right now, no details on what 2+ means, but, the target market is market data distribution (and this was discussed in the slides) so I would assume that they've thought about this. 5ns is additional functionality available in previous products (http://exablaze.com/exalink-50 http://exablaze.com/exalink-50) which is also in this product. From the sides, the idea is that they can set up multiple broadcast groups with the 5ns device to distribute market data, and then aggregate responses together with with 110ns (or in some cases 100ns) device, all in one box.

myrandomcomment 12y ago
So the exalink-50 number is as a Layer1 switch. So the comment "can setup multiple broadcast groups" sounds like the Cisco Nexus 3548 Warp Mode. The market did not take to that so not sure how this is going to change that. Like I said, I have not reviewed all the data yet. An important step for verification for all this is to put it in the hands of David Newman for a test, either paid via his private company Network Test, or publicly via Network World. A testing house like EANTC would also be great. As you can might have guessed from my post I know a bit about this world (I am not involved anymore). At this point it is more of a historical curiosity on my part more then anything (I still have a number of friends that run HFT networks at the banks). Given what I saw in testing and claims during the early HFT rush I always want to see 3rd party testing from a reputable testing house showing a real market data feature set being used. Here is a starting list: 1. BGP running 2. Mulitcast w/PIM 3. IGMP snooping 4. ACL configured 5. 1 port to max port of switch fanout. 6. 1% to 100% line rate on the feed. 7. Packet size ranges, both fixed and mixed. 8. Join/leave time. 9. Max groups joined.
- deadgrey19 12y ago
  Agreed. Independent verification is definitely needed. Although I think that this is why they've announced this at STAC which does this (https://stacresearch.com/about https://stacresearch.com/about). I've moved over to academia now, but I came from tan HFT prop-trading background. We would have jumped at the chance to get one of these in our hands, but this was 3 years ago now. Can I ask why you'd want BGP running in a private LAN? This seems like an odd thing to do, especially in Colo environments.