4 ms·

How to Run QwQ-32B effectively

• danielhanchen 2y ago
  Qwen released QwQ-32B - a reasoning model with performance comparable to DeepSeek-R1 on many benchmarks. However, people have been experiencing infinite generations, many repetitions, <think> token issues and finetuning issues. I hope this guide will help debug and fix most issues!
  • kiratp 2y ago
    I hope this amount of effort means multi-GPU support is coming soon (saw you mention “this week” :) ) so we can tune QwQ on a single node!
    • danielhanchen 2y ago
      Yes!! Technically it's already in Unsloth :) I'm still trying to make the frontend for running multi GPU better!
• deleted 2y ago
  [deleted]