4 ms·
If you're bound by memory throughput, you can't go beyond a 2x speedup (there's 1/2 as much data to move in an 8b format, whether it's in vectors or matrices do
by stephencanon 9y ago
If you're bound by memory throughput, you can't go beyond a 2x speedup (there's 1/2 as much data to move in an 8b format, whether it's in vectors or matrices doesn't matter). I still don't see any reasonable expectation for 4x.
- dnautics 9y agoif your matrix contents are static during your rate-limiting step (as they are for most DL applications) your FLOPs scale with O(n^2) relative to your memory throughput on your vector component. [a b, c d] dot [e, f] is four multiplies [a b c, d e f, g h i] dot [j, k, l] is nine multiplies.
- deleted 9y ago[deleted]