3 ms·
I run convnet-benchmarks, and also work at Facebook. Unlike what the public perception is, NVIDIA, Facebook and other companies collaborate very very closely.
by smhx 11y ago
I run convnet-benchmarks, and also work at Facebook.
Unlike what the public perception is, NVIDIA, Facebook and other companies collaborate very very closely. NVIDIA's latest R3 release has FFT-based convolutions inspired by Facebook's work, and Facebook integrated some of the optimization suggestions for fbfft that NVIDIA gave. Further optimizations on the FFT approach (like tiled-FFTs) have been suggested and prototyped at FB and carried over by NVIDIA. The relationship is more of a deep collaborative effort.
At the end of the day, we want cuDNN to win, as it is a baseline system for all software.
However, if you want to do research, you want to have open-source code, because you often want to do funky changes to the base algorithm. NVIDIA falls short on this, and we at Facebook pledged to always keep our code open source (bitten by the fact that we had to start with closed-source blackboxes for our initial FFT work).
- varelse 11y agoSure, but most of the DNN people are just running glorified AlexNets and GoogleNets and not really looking deeply under the hood (I speak from 1st hand experience). For Caffe is open source. And cuDNN makes it fast. And if it doesn't, just jump and down like the above report did, and it will be fast in the next release. IMO if you're doing stuff for which cuDNN isn't good enough, you're probably capable of writing your own kernels, and that's awesome. Welcome to the Deep Learning equivalent of the 1%. Meanwhile, incorporate cuDNN into your production workflow to maximize comparative advantage. Or don't, whatever...