3 ms·
30-40 at 64k context, but it's a mixture of experts model. A 70b dense model is slower Qwen coder 30b Q4 runs 40+.
by data-ottawa 1y ago
30-40 at 64k context, but it's a mixture of experts model.
A 70b dense model is slower
Qwen coder 30b Q4 runs 40+.