4 ms·
It seems very good at understanding human language clues even in a 4-bit (Q4_K_S) model, similar in feel to E4B but a great incremental improvement. Interestin
by LarsKrimi 4mo ago
It seems very good at understanding human language clues even in a 4-bit (Q4_K_S) model, similar in feel to E4B but a great incremental improvement.
Interesting for my 8GB VRAM system, but the system RAM requirement seems to balloon quickly, and it starts misspelling words. Also token/s drops off quickly it seems