4 ms·
Neural networks are commonly evaluated with matrix multiplication. A matrix multiply is orders of magnitude faster if all three matrices fit in your caches. So
by ip26 4y ago
Neural networks are commonly evaluated with matrix multiplication. A matrix multiply is orders of magnitude faster if all three matrices fit in your caches. So you want to tune the matrix size to the hardware.
- WithinReason 4y agoIn addition, once you did that (tuned your hardware to a matrix size) it can be inefficient to use smaller matrices. So even if you can make a network that uses smaller components the HW will not gain the corresponding speedup.
- zasdffaa 4y agoMatMult = the dot product, effectively. I can't believe that allowing smaller vectors is difficult or even non-trivial in hardware.
- svnt 4y agoYou’re assuming caches know anything about dot products.
- zasdffaa 4y agoThat seems a non-sequiteur to me. Can you elaborate?
- WithinReason 4y agoGPUs are a good example: SIMD means all 128 threads do the same work. If only 1 thread has work to do the other 127 threads still take up resources.
- zasdffaa 4y agoAh, right, gotcha
- deleted 4y ago[deleted]