4 ms·
That's not a minimum, just what the author had on hand. I can get 65B and 70B running just fine with exllama on 2 3090s.
by trifurcate 3y ago
That's not a minimum, just what the author had on hand. I can get 65B and 70B running just fine with exllama on 2 3090s.