3 ms·
I'd be interested in other people's recommendations as well. Personally I'm mostly using openchat with q5_k_m quantization. OpenChat is imho one of the best 7B
by wongarsu 2y ago
I'd be interested in other people's recommendations as well. Personally I'm mostly using openchat with q5_k_m quantization.
OpenChat is imho one of the best 7B models, and while I could run bigger models at least for me they monopolize too many resources to keep them loaded all the time.