2 ms·
Is it because they will have to communicate back errors during training? I forgot that training these models is more of a global task than proteins folding. In
by newswasboring 6y ago
Is it because they will have to communicate back errors during training? I forgot that training these models is more of a global task than proteins folding. In that sense this is less parallelizable over the internet.
- londons_explore 6y agoYes, and also activations if your GPU is too small to fit the whole model. The minimum useful bandwidth for that stuff is a few gigabits...