3 ms·
We don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison
by cellularmitosis 1mo ago
We don’t know what quantization level was used for the weights or the kv cache for you or for parent poster, so this is probably an apples to oranges comparison.
- spacebacon 1mo ago[dead]