3 ms·It depends on the context window, but my 3090 gets ~60/s on smaller windows.by cellis 3y agoIt depends on the context window, but my 3090 gets ~60/s on smaller windows.visarga 3y agoI get 50-60t/s on Mistral 7B on 2080 Ti