4 ms·
> 100 GPU's to get a difference on fourth decimal digit! So now we know what not to do. That's valuable. So what if it's not the best theoretical paper. This
by grizzles 8y ago
> 100 GPU's to get a difference on fourth decimal digit!
So now we know what not to do. That's valuable.
So what if it's not the best theoretical paper. This screed rehashes criticisms that are well known among researchers in the field. Overall it reads like a kind of egotistical hit piece. Personally I'm glad Uber published it.
- throwaway080383 8y agoYeah, I'm not a deep learning researcher, but I didn't understand the incredulity at 100 GPU's. Surely Uber has that many lying around, so why not overkill on the hardware when testing? This leaves no doubt that the feature is worthless for the given task, and does not leave open the question of whether more hardware could produce better results.
- HelloNurse 8y agoIt's a genuine scientific question: adding a useless feature slows down NN training, but how much? Is the impact as negligible as can be expected? There's no reason to spend lots of brain cycles on low-value theory and possibly wrong estimates instead of lots of GPU cycles on a conclusive experiment.
- stigsb 8y agoIf training requires N operations, using more GPUs (that otherwise would idle) simply means you finish (and iterate) faster.