3 ms·
You should use FFT's for the convolution, see http://arxiv.org/abs/1312.5851 http://arxiv.org/abs/1312.5851 Also, while the original Blas was in Fortran, you p
by im3w1l 12y ago
You should use FFT's for the convolution, see http://arxiv.org/abs/1312.5851 http://arxiv.org/abs/1312.5851
Also, while the original Blas was in Fortran, you probably want OpenBlas, which is in C. Or maybe Intels MKL.
- ajtulloch 12y agoFWIW, FFT-based convolutional methods aren't better for a lot of common sizes (including a bunch of AlexNet/VGG/GoogLeNet sizes). http://arxiv.org/abs/1412.7580 http://arxiv.org/abs/1412.7580 has a pretty thorough evaluation of im2col + gemm vs fused gemm (cudnn) vs fft.
- im3w1l 12y agoThanks for the article tip hadn't seen it. But I don't agree with your conclusion. In fig 1-6, you should look at the last column. Because really the other channel numbers are quite low. Knowing that, the space domain algo, only really has a chance at 3x3 filters which is quite small. And since they claim fbfft was even faster than cufft, the situation should look even better for fft. EDIT: And they are about tied in the 3x3, 64 case.
- gcr 12y agoThe current trend in convolutional neural networks seems to be moving toward more convolutions with smaller kernels. This year's second-place ILSVRC winner (VGG) is essentially a giant stack of 19 convolution layers with super tiny kernels, sandwiched between nonlinearities and pooling. "To reduce the number of parameters in such very deep networks, we use very small 3×3 filters in all convolutional layers" http://arxiv.org/pdf/1409.1556.pdf http://arxiv.org/pdf/1409.1556.pdf
- gcr 12y agoThe article actually touches on this issue a bit. "The main competitor to the GEMM approach is using Fourier transforms to do the operation in frequency space, but the use of strides in our convolutions makes it hard to be as efficient."
- gone35 12y agoFor 'small' workloads, it's not uncommon to find situations where direct computation might make more sense. The author touches on that a bit, as others have pointed out.
- shepardrtc 12y agoAccording to Github, 51.6% of the code in OpenBLAS is Fortran: https://github.com/xianyi/OpenBLAS https://github.com/xianyi/OpenBLAS