8 ms·

True - they are workhorses. Not super bright, but good enough for lots of everyday tasks. I've found sweet spot to be turning thinking off, as it adds small or

True - they are workhorses. Not super bright, but good enough for lots of everyday tasks. I've found sweet spot to be turning thinking off, as it adds small or no value, while increasing the token count and waiting time. Last 27B I used was https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-GGUF https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-GGUF - specifically post-train adapted a bit to run with thinking off. I saw today the 35B-A3B MoE from the same HF acc is out, downloading that rn to try.

kroaton 3mo ago
Please don't use that garbage. Just use the base Qwen models or Nex/Orinth, as those are the only properly post-trained finetunes. The Qwopus models are marketing.
- aand16 3mo ago
  Can you expand on why Qwopus is not recommended and what "Nex/Orinth" brings to the table?
- ljosifov 3mo ago
  I usually doubt the 'small dataset tuned' variants. B/c ages ago (in the NN prehistory) I've done some NN training, and appreciate how hard it is to improve in general, and how easy it is to ruin a model in general while targeting a small dataset (LoRA-s are ok, that's different). That model/quant was the most recent one I was trying. But could not really use any of them, as the combo model + llama-server ground to a halt even at small context depth sizes on the amd gpu. Yesterday I finally found a good combo! So writing this for the benefit for anyone that may have the same h/w. Got around to /GOAL search for something better for the h/w (amd 7900xtx), and pi agent found a new best that actually seems it will be useful for real. As the 40 tok/s speed starts dropping only at 260K context depth?? Served by hipfire from this repo https://github.com/Kaden-Schutt/hipfire https://github.com/Kaden-Schutt/hipfire, that worked the best got on llama-benchy: | Context | Wall Time | PP (t/s) | TG (t/s) | |:-------:|:---------:|:--------:|:--------:| | 1,024 | 17.3s. | 660 | 40.8 | | 8,192 | 29.3s | 606 | 40.8 | | 32,768 | 70.5s | 599 | 40.7 | | 65,536 | 126.5s | 594 | 40.6 | | 131,072 | 235.5s | 591 | 39.0 | | 196,608 | 349.0s | 591 | 40.2 | | 260,000 | 451.3s | 594 | 37.2 | This is - hipfire daemon dflash_mode auto, serve qwen3.6:27b --kv-mode asym2. Models: hipfire pull qwen3.6:27b # 14 GB MQ4 quant, qwen3.6-27b.mq4 hipfire pull qwen3.6:27b-draft # 0.92 GB DFlash draft, qwen36-27b-dflash-mq4.hfq

kroaton 3mo ago

Please don't use that garbage. Just use the base Qwen models or Nex/Orinth, as those are the only properly post-trained finetunes. The Qwopus models are marketing.

aand16 3mo ago
Can you expand on why Qwopus is not recommended and what "Nex/Orinth" brings to the table?
ljosifov 3mo ago
I usually doubt the 'small dataset tuned' variants. B/c ages ago (in the NN prehistory) I've done some NN training, and appreciate how hard it is to improve in general, and how easy it is to ruin a model in general while targeting a small dataset (LoRA-s are ok, that's different). That model/quant was the most recent one I was trying. But could not really use any of them, as the combo model + llama-server ground to a halt even at small context depth sizes on the amd gpu. Yesterday I finally found a good combo! So writing this for the benefit for anyone that may have the same h/w. Got around to /GOAL search for something better for the h/w (amd 7900xtx), and pi agent found a new best that actually seems it will be useful for real. As the 40 tok/s speed starts dropping only at 260K context depth?? Served by hipfire from this repo https://github.com/Kaden-Schutt/hipfire https://github.com/Kaden-Schutt/hipfire, that worked the best got on llama-benchy: | Context | Wall Time | PP (t/s) | TG (t/s) | |:-------:|:---------:|:--------:|:--------:| | 1,024 | 17.3s. | 660 | 40.8 | | 8,192 | 29.3s | 606 | 40.8 | | 32,768 | 70.5s | 599 | 40.7 | | 65,536 | 126.5s | 594 | 40.6 | | 131,072 | 235.5s | 591 | 39.0 | | 196,608 | 349.0s | 591 | 40.2 | | 260,000 | 451.3s | 594 | 37.2 | This is - hipfire daemon dflash_mode auto, serve qwen3.6:27b --kv-mode asym2. Models: hipfire pull qwen3.6:27b # 14 GB MQ4 quant, qwen3.6-27b.mq4 hipfire pull qwen3.6:27b-draft # 0.92 GB DFlash draft, qwen36-27b-dflash-mq4.hfq