3 ms·
Build your systems around GEMM/blas. Every vendor will give you a fast GEMM, and you'll be set for basically all the architectures that are coming out.
by paulsutter 9y ago
Build your systems around GEMM/blas. Every vendor will give you a fast GEMM, and you'll be set for basically all the architectures that are coming out.
- arcanus 9y agoExcept that not all problems in computation are GEMMs. CNNs in Machine learning certainly are, but many 'real' systems cannot be posted in such a manner. In supercomputing this is the problem with using high performance linpack for benchmarks, which typically exceeds actual scientific codes by an order of magnitude in terms of floating point operations per second.
- paulsutter 9y agoYes but to the extent you can, it's an easy win. I switched to a GEMMable method for a preprocessing step today based on the Volta and recent TPU news. Hopefully Tensorflow XLA or other optimization frameworks could solve this problem in a more general way in the medium term: http://www.kdnuggets.com/2017/04/deep-learning-virtual-machine-rule-all.html http://www.kdnuggets.com/2017/04/deep-learning-virtual-machi...