5 ms·

GEMVs can be used to do GEMMs

• atq2119 8y ago
  Yes. The difference for the purpose of this discussion is in the dispatch (data movement) cost per useful operation. Both GEMV and GEMM can be described as performing (m,k,n) matrix multiplication of an mxk matrix by a kxn matrix. GEMV is simply the case n=1. The number of useful operations is m * k * n, while the size of the input data is m * k + k * n. So a (4,4,4) GEMM does 64 useful operations while moving 32 input values. Implementing the same GEMM as 4xGEMV also does 64 useful operations, but at the cost of moving 20 input values per GEMV, or 80 overall. That's where the benefit of hardware GEMM comes from. Computer graphics doesn't require high throughput of matrix-matrix multiplication. You might need a few matrix-matrix multiplications to set up your transformation matrices, but you do that once for matrices that are then applied to many vertices, so there's not much to be gained by optimizing those. The high throughput matrix-vector multiplies happen on the GPU, but you don't need GEMM for that and so GPUs traditionally didn't offer it. I guess you could argue that if you multiply one matrix by many vectors, for processing many vertices of a model, then you do in fact have an implied GEMM if you group your vertices accordingly. It seems that for some reason, the computer graphics folks never quite saw it that way, maybe because you also do stuff like animation blending which breaks the GEMM analogy.