3 ms·
> which seems to be the main reason for a speedup in this model I wouldn't agree with this as there are substantial perf improvements on GPTQ models of ca. 20%
by qeternity 3y ago
> which seems to be the main reason for a speedup in this model
I wouldn't agree with this as there are substantial perf improvements on GPTQ models of ca. 20%.