3 ms·
> we have successfully used both RoCE and InfiniBand clusters for large, GenAI workloads (including our ongoing training of Llama 3 on our RoCE cluster) without
by latchkey 3y ago
> we have successfully used both RoCE and InfiniBand clusters for large, GenAI workloads (including our ongoing training of Llama 3 on our RoCE cluster) without any network bottlenecks.
Interesting dig on IB. RoCE is the right solution since it is open standards and more importantly, available without a 52+ week lead time.
- loeg 3y agoYeah, and RoCE isn't single vendor. I'm not sure IB scales to the relevant cluster sizes, either.
- anonymousDan 3y agoIs NVLink just not scalable enough here?
- loeg 3y agoI don't know. I haven't actually worked with IB in this specific space (or since before Nvidia acquired MLNX). My experience with RoCE/IB was for storage cluster backend in the late 2010s.