4 ms·
It isn't. It's just that 4bit implementations are really bad. Exllama has shown how properly optimized and fused kernels can perform.
by qeternity 3y ago
It isn't. It's just that 4bit implementations are really bad.
Exllama has shown how properly optimized and fused kernels can perform.