2 ms·
If we go with AA's benchmarks Qwen 3.8 27B is already slightly below Luna level which is in itself impressive, but with this compression it should be just sligh
by ctolsen 8d ago
If we go with AA's benchmarks Qwen 3.8 27B is already slightly below Luna level which is in itself impressive, but with this compression it should be just slightly more below Luna level and could run on my old GTX 1070 that I'm now tempted to fire up. That's kinda nuts even allowing for small-model problems that I'm sure I'd see quite clearly.