3 ms·
Just to throw out there - I haven't had any issues with exllama personally, and it's a lot faster last I checked.
by declaredapple 3y ago
Just to throw out there - I haven't had any issues with exllama personally, and it's a lot faster last I checked.
- brucethemoose2 3y agoYeah this is what I use as well. The arbitary quantization is just great, as is everything else. Its not easy to install though, and big dGPU only.