thanks for the info! I have very limited knowledge of these things but I did compare matmul with PyTorch using GPU vs not and there is a dramatic improvement.

thanks for the info! I have very limited knowledge of these things but I did compare matmul with PyTorch using GPU vs not and there is a dramatic improvement. So even if its not fully optimised yet, it's still a huge bonus to have this available. If it could be improved another 4-6x that would be stupendous. (For context: am observing an 8000x8000 matrix takes ~1s to multiply with CPU and 5ms to multiply with GPU).