7 ms·
I only know of Mellanox for their 10 gigabit ethernet cards. Does anybody know if this is a good or bad thing for Mellanox?
by penagwin 8y ago
I only know of Mellanox for their 10 gigabit ethernet cards. Does anybody know if this is a good or bad thing for Mellanox?
- baybal2 8y agoPlumpy "enterprise" customers. They still are the company to go for infiniband, but infiniband it lost much of its appeal to non true supercomputing tasks. Ethernet nowadays can do RDMA, soft guarantees on latency, in-order and reliable delivery at lower costs, and an option to reuse existing L2 networks. Mellanix has squeezed the infiniband cow dry.
- adev_ 8y ago> Ethernet nowadays can do RDMA, soft guarantees on latency, in-order and reliable delivery at lower costs, and an option to reuse existing L2 networks. Mellanix has squeezed the infiniband cow dry. And who did the Ethernet RDMA protocol (RoCEv1/v2) and sell the RDMA compatible NIC that everyone use in the HPC world currently ? Tip: It is starting by Mella too.
- chx 8y agoRDMA is an interesting point because QLogic was doing some stuff then Cavium bought them then Marvell bought them. And then there's the Emulex-Avago-Broadcom chain. The entire market is converging into a few major players.
- SEJeff 8y agoAnd yet an ethernet frame, by design, is larger than an infiniband frame (think layer 2). When it comes down to node to node latency, given perfectly equal silicon, infiniband will still be faster.
- jabl 8y agoI think the minimum size of an IB packet with no payload is 26 octets, vs. 64 octets for an eth packet. So sure, a difference of 38 octets, but at, say, 100 Gbit/s, that's less than a nanosecond difference, much much less than the IB vs. ethernet latency difference. So I think you'll have to look somewhere else for information. I have no idea what it is, actually. Some ideas that may or may not matter (or might not even be correct): - IB is a couple of decades younger, so could benefit from knowledge how to do fast protocols. (Not an explanation per se) - Simpler forwarding. In IB the subnet manager gives out the LID's that are used for routing withing a subnet. They are shorter than an eth MAC (16 vs. 48 bits), so the lookups circuit in the switches can be smaller and faster(?), and also since the LID's are assigned by the subnet manager rather than being burned at the factory, they can be distributed taking into account the subnet topology, allowing switches to use LID Mask Count (LMC) filtering. Similarly, all routes within a subnet are calculated statically a priori by the subnet manager (load balancing among multiple paths is only static round robin, not dynamical load dependent), and don't have to be calculated on the fly by the switches. - FEC rather than retransmission in case of corruption.
- baybal2 8y agoSure, IB is simply is a superior fabric for its niche. For everything else, RDMA on Ethernet buys you with ability to reuse your L2, and this matters way way more to people running DC businesses than anything else.
- dogma1138 8y agoOne out of only 2 vendors for InfiniBand which is quite important for HPC especially in the Top 500. They also have some sort of a parallel VLIW CPU architecture that they've been trying to get off the ground for a while now called TILE/TILE64 so that might also play into things. However since NVIDIA opened their offices in Israel a while ago they might simply be looking for an acquihire since Mellanox is a fabless semi chip maker it kinda fits that also.
- dkersten 8y agoOh. I've known about TILE64 back when Tilera created it (I spoke to their VP of something or other on the phone once, I worked in telecoms at the time and we were interested in seeing if their hardware would help accelerate something we worked on. We never went forward with it though), I hadn't realised Mellanox owned it now. Looking at wikipedia, Tilera was bought by EZChip, which was bought by Mellanox.
- GordonS 8y agoThat'd be a lot of money for an aquihire. Are you aware of any other aquihire's in the billion dollar range?
- dogma1138 8y agoDon't think so, but they do have like 3000 employees mainly in Israel and design a wide range of ASICs I doubt that they got them for the InfiniBand and their interconnect business alone.
- wtallis 8y agoI think Mellanox is moving away from the Tilera architecture in favor of re-using the mesh interconnect with ARM cores, as in their BlueField chip.
- dogma1138 8y agoCould be, but their mesh fabric might be useful for some multi-GPU configurations especially if NVIDIA goes into chiplets, I don't know if it's better than NVLINK or not but since NVLINK looks to be pretty much PCIe with a lot of the overhead stripped out of it it just might be.
- mikehollinger 8y agoThe 10g NICs are toys. ;-) Mellanox provides high speed (40/56/100/200 gigabit) switches and adapters. If you think 10g fiber is big, just wait... I’ve worked on several projects with them, and found they generally do a good job of feeding the beast when the OS and driver’s are properly tuned. I’m not informed enough to call good or bad, but will instead say it’s interesting, especially in the HPC space (and the emerging AI space).
- arminiusreturns 8y agoCan confirm, running 100g mellanox and getting ready to move to 200. They are the best game in town for the price point to performance/reliability/support ratio.
- edoo 8y agoLast I was looking you could get a 40GB card for around $300. Very impressive price that kind of speed in a home office.
- TheValeyard 8y agoI use Mellanox ConnectX-5 Dual QSFP+ 100Gbit Ethernet cards in my OpenStack private cloud at my business. Mellanox has been instrumental in running flash Ceph arrays by pushing the speed envelope beyond what the Intel/Cisco's of the world are doing. Mellanox has also embraced bringing RDMA to things like Ceph and working with the broader vendor ecosystem like Red Hat for using this in production. I hope Nvidia doesn't taint the good reputation of this company.