3 ms·
The problem exists with the default 7B model. I don't know if different quantizations would fix the problem. The 2B model is fine, though.
by LorenDB 3y ago
The problem exists with the default 7B model. I don't know if different quantizations would fix the problem. The 2B model is fine, though.