3 ms·
thanks for sharing. are some local models better than others? can small models work well or do you want 8B+?
by Tokumei-no-hito 1y ago
thanks for sharing. are some local models better than others? can small models work well or do you want 8B+?
- vunderba 1y agoSo in my experience smaller models tend to produce worse results BUT I actually got really good transcription cleanup with CoT (Chain of Thought models) like Qwen even quantized down to 8b.
- dragonwriter 1y agoI think the 8B+ question was about parameter count (8 billion+ parameters), not quantization level (8 bits per weight).
- vunderba 1y agoYeah I should have been more specific - Qwen 8b at a 5_K_M quant worked very well.