I did pretty much the same in HLSL just a few days ago. Here’s one idea to try. You don’t actually need 3 tiles in groupshared memory. With some care, it’s po

I did pretty much the same in HLSL just a few days ago. Here’s one idea to try. You don’t actually need 3 tiles in groupshared memory. With some care, it’s possible to go with 2 tiles there. One tile is loaded from the first source matrices. Another tile is accumulator for the output matrix tile. And then in the main function of the compute shader load values for the second source matrix from the global memory, and update all affected elements of the output tile with these mad() instructions. Shader model 5.0 limits amount of group shared memory to 32kb, and that streaming trick allows to push to the limit, with 64x64 tiles. I think I got about 15% performance win from that trick on some of the GPUs I’m testing my software.