3 ms·
There's a hell of a lot of matrix multiplications going on in AI/ML. Shaving off a few milliseconds for ultra large matrices could save power and money for the
by UncleOxidant 3y ago
There's a hell of a lot of matrix multiplications going on in AI/ML. Shaving off a few milliseconds for ultra large matrices could save power and money for the big players like Google, Microsoft/OpenAI and Meta.
- imtringued 3y agoThe bottleneck in those is not the arithmetic operation but the memory bandwidth once you have to spill your matrix out of SRAM. As it stands right now, it is actually better to have a slower algorithm that uses the local memory more efficiently.
- amanda99 3y agoThose are by and large many small matrices being multiplied, these results have nothing to do with that.