4 ms·
I was going to try these models with Ollama. Did you use a small number of bits/quantization?
by mark_l_watson 3y ago
I was going to try these models with Ollama. Did you use a small number of bits/quantization?
- LorenDB 3y agoThe problem exists with the default 7B model. I don't know if different quantizations would fix the problem. The 2B model is fine, though.