4 ms·
How do they compare to their original quants on ollama like q4_K_S?
by arnaudsm 2y ago
How do they compare to their original quants on ollama like q4_K_S?
- tcdent 2y agoThese undergo additional fine tuning (QLoRA) using some or all of the original dataset, so they're able to get the weights to align to the nf4 dtype better, which increases the accuracy.