3 ms·
Thank you, that's exactly what I was looking for, specific info on perf.
by rnk 4y ago
Thank you, that's exactly what I was looking for, specific info on perf.
- anentropic 4y agoI think the GPU performance for inference is probably limited currently by immaturity of PyTorch MPS (Metal) backend before I found the repo above I had a naive attempt to get llama running with mps and it didn't "just work" - bunch of ops not supported etc