5 ms·
Note that GPUS are almost always multiplying 4x4 matrices. Sure they multiply many matrices together but each is 4x4. The 4^3 complexity isn't at all an issue (
by AnotherGoodName 6y ago
Note that GPUS are almost always multiplying 4x4 matrices. Sure they multiply many matrices together but each is 4x4. The 4^3 complexity isn't at all an issue (64 steps) and the constant time for some of the n^2.x methods is high.
- nicoburns 6y agoPresumably this only applies to GPUs being used for graphics, not GPUs being used for ML?
- shaklee3 6y agoThis is not true. What is your source?
- antixk 6y agoI guess OP meant that each tensor core in the latest NVIDIA GPUs fundamentaly performs 4x4 matrix multiplication [0]. [0] https://nla-group.org/2020/07/21/numerical-behaviour-of-tensor-cores/ https://nla-group.org/2020/07/21/numerical-behaviour-of-tens...
- shaklee3 6y agoThat's also not true. They do 4x4, 8x8, and 16x16.
- AnotherGoodName 6y agoWhich is fine for the point being made here. There's no issue with n^3 for small matrices.