3 ms·
if your matrix contents are static during your rate-limiting step (as they are for most DL applications) your FLOPs scale with O(n^2) relative to your memory th
by dnautics 9y ago
if your matrix contents are static during your rate-limiting step (as they are for most DL applications) your FLOPs scale with O(n^2) relative to your memory throughput on your vector component.
[a b, c d] dot [e, f]
is four multiplies
[a b c, d e f, g h i] dot [j, k, l]
is nine multiplies.
- deleted 9y ago[deleted]