2 ms·
I've been waiting to see a response to Nvidia's NVLink switching in the hubbub from AMD like this, as NVLink is what makes them viable across large clusters. I
by bastard_op 3y ago
I've been waiting to see a response to Nvidia's NVLink switching in the hubbub from AMD like this, as NVLink is what makes them viable across large clusters.
I've still not seen what a physical NVLink nic or switch actually consists of either in production beyond flashy marketing stating a 400/800G phy, I assume it's just a proprietary optimized Infiniband, but it would be nice to get some more info from some industry insiders building these "supercomputer" clusters, as it'll be a while before they start hitting ebay off-lease I think.
- namibj 3y agoThe older generations had substantial docs; they were iirc basically MPLS on their L2. With a bunch of e.g. remote atomic verbs that even worked through to POWER9 hosts, great for throughout on fetch_and_add heavy (almost) wait-free data structures. Such as even just bump-allocating yourself a remote destination buffer to deliver your message to (in LLM context, this could e.g. be single-token granular output delivery from an MoE router to which GPU hosts that expert. The router spits out a ragged Tensor, and those parts can be delivered efficiently.).