4 ms·
Does gcp have the worst networking for gpu training though?
by dangoodmanUT 2y ago
Does gcp have the worst networking for gpu training though?
- dweekly 2y agoFor TPU pods they use 3D torus topology with multi-terabit cross connects. For GPU, A3 Ultra instances offer "non-blocking 3.2 Tbps per server of GPU-to-GPU traffic over RoCE". Is that the worst for training? Namely: do superior solutions exist?