3 ms·
> . The dynamic Q3 is a good balance between accuracy and size keeping I'm having a hard time understanding how you find any sort of accuracy in Q3, when I use
by embedding-shape 1mo ago
> . The dynamic Q3 is a good balance between accuracy and size keeping
I'm having a hard time understanding how you find any sort of accuracy in Q3, when I use it with BF16 and it's hardly usable due to drastic hallucinations and inability for system prompt following. But, if it works for you, that's pretty good! Guess I'm jealous :)
- petruspennanen 1mo agoHmm we must be doing something different, I think the official accuracy measurement for the Q3 is more than 90%. Maybe some bug affecting your test?