4 ms·
We have distributed implementations that allow us to use more than 8 GPUs to train a single network. These machines have 8 cards each, but we aren't limited to
by spiantino 11y ago
We have distributed implementations that allow us to use more than 8 GPUs to train a single network. These machines have 8 cards each, but we aren't limited to a single machine (though having more GPUs connected via PCI than via Ethernet helps)
- mrlase 11y agoAny particular reason you don't have Infiniband on these for interconnects? Having this chassis with Infiniband and a local disk would be a dream if the manufacturing cost is right as we scale up in our local datacenter where I'm at.
- vonnik 11y agoThat's interesting. But from what I understand, the problem isn't getting many GPUs to train a single network -- the problem is to get them to scale linearly; i.e. to get each additional GPU to accelerate the process as much as the last. Throwing boxes at neural net training is easy, but people run into serious plateaus scaling GPUs.