3 ms·

Have you compared it with 8-bit QwQ-17B? In my evals 8 bit quantized smaller Qwen models were better, but again evaluating is hard.

• redrove 2y ago
  There’s no QwQ 17B that I’m aware of. Do you have a HF link?
  • xiphias2 2y ago
    You're right, sorry...I just tested Qwen models, not QwQ, I see QwQ only has 32B.
    • redrove 2y ago
      No worries, QwQ is the thinking model from Qwen, it’s a common misconception. I think they should’ve named it something else.