3 ms·
interestingly, i am also on M2 Max, and i get ~66 tok/s in LM Studio on M2 Max, with the same 131072. I have full offload to GPU. I also turned on flash attenti
by rt1rz 1y ago
interestingly, i am also on M2 Max, and i get ~66 tok/s in LM Studio on M2 Max, with the same 131072.
I have full offload to GPU. I also turned on flash attention in advanced settings.
- hrpnk 1y agoThank you! Flash attention gives me a boost to ~66 tok/s indeed.