4 ms·
If you can computationally afford it, 7b-q5_K_M is a way better choice. Default :7B goes to q4_0 which might give you subpar results.
by syntaxing 2y ago
If you can computationally afford it, 7b-q5_K_M is a way better choice. Default :7B goes to q4_0 which might give you subpar results.
- xyc 2y agoTried 7b q5 with some RAG tasks. Seems quite impressive.