3 ms·

> Is not far at all from proprietary models if you give it tools, skills and agents etc, I use Qwen 3.6 27B, the dense version of this model which is slightly

> Is not far at all from proprietary models if you give it tools, skills and agents etc, I use Qwen 3.6 27B, the dense version of this model which is slightly better. I don't agree that it's close at all. Maybe for some small, easy tasks, but not for working on real codebases. It's amazing for something I can run at home, but the difference between it and Opus or GPT-5.5 is huge.

tedivm 6mo ago
I've had the opposite experience, and have built multiple fantastic applications with Qwen3.6 27b. What quantization have you tested with?
- hedgehog 6mo ago
  Similarly I haven't seen Qwen 27B as remotely competitive with Opus, at least Q4 hooked up to Claude Code. What harness are you using?
- trilogic 6mo ago
  As funny as it may sound a q4_k_m well converted and quantized properly (and finetuned, impereative) would do the job. The 27b it may be good but is heavy, it burns the hardware. I personally prefer the 397B if I am stucked and can´t progress, it can still run with 7 tps. Now with the Mtp (multitoken prediction) it nearly double the speed ( reached 82tps today with the 35b 100000ctx). I recommend it you give it a try.
trilogic 6mo ago
Really, how so? Because we work with codebases daily, can you tell us a concrete example! In our case we work in consumer hardware (ish), 10 million ctx (1 million output, 1 million input proven, sometimes it loops or breaks at over 500k ctx byt at ~17tps linear). IT can read the full codebase, unleash agents, and write in disk editing and patching files creating a full app in 3-4 minutes. IT can do Web search and Rag pretty fast, it understands and fix the user query, sys prompts and adapt/fix them if needed on the fly. I am wondering what more do you do?
- trilogic 6mo ago
  Edit: Forgot to mention that it can process images and pdf, and 100s of other files, it can even create presentations in code or mermaid, svg, charts js etc. Here a basic version of it: https://hugston.com/chat https://hugston.com/chat
  - rspoerri 6mo ago
    how do you do 1mio context with qwen3.6 27b, that only supports 256k? and what hardware would you run that on? 2 * 3090 is afaik currently at max 256k context.
    - omneity 6mo ago
      You can increase the context window beyond its max trained context using RoPE scaling[0] which will require more VRAM. But you can increase your context window for the same VRAM by quantizing the KV cache with FP8 (double the context) or TurboQuant (more than double)[1]. 0: https://medium.com/@leannetan/extending-context-length-with-hugging-faces-transformers-6b04db05b39a https://medium.com/@leannetan/extending-context-length-with-... 1: https://docs.vllm.ai/en/latest/features/quantization/quantized_kvcache/#example-quantize-llama-attention-kv-cache-to-fp8 https://docs.vllm.ai/en/latest/features/quantization/quantiz...
    - trilogic 6mo ago
      We managed to increase the ctx for whatever llm model that is GGUFED, here the experimental tests: https://www.reddit.com/r/Hugston/ https://www.reddit.com/r/Hugston/
    - nyrikki 6mo ago
      You can get all the Qwen 3.x models up to ~1 million tokens using YaRN with llama.cpp.[0] Personally I am using `--no-context-shift` and feeding in context back in on failure at the harness level. I have 2x1080ti + 1xTitanV that have a full 262,144 tokens context on 262,144 tokens with `-sm tensor` at 62.04 t/s which isn't so bad. But I also have a 1x3090 running unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL at 41.89 t/s but with only 130k context, but if you have a modular programming style both work pretty well. But play with YaRN if you really need it. [0]https://qwen.readthedocs.io/en/v3.0/run_locally/llama.cpp.html https://qwen.readthedocs.io/en/v3.0/run_locally/llama.cpp.ht...
0xbadcafebee 6mo ago
> not for working on real codebases You don't pick just one model to "work on real codebases". You use a very advanced model to plan, and a not-very-advanced, cheaper, faster model to execute planned tasks. This saves money and speeds up work. This is the guidance from Anthropic & OpenAI.

tedivm 6mo ago

I've had the opposite experience, and have built multiple fantastic applications with Qwen3.6 27b. What quantization have you tested with?

hedgehog 6mo ago
Similarly I haven't seen Qwen 27B as remotely competitive with Opus, at least Q4 hooked up to Claude Code. What harness are you using?
trilogic 6mo ago
As funny as it may sound a q4_k_m well converted and quantized properly (and finetuned, impereative) would do the job. The 27b it may be good but is heavy, it burns the hardware. I personally prefer the 397B if I am stucked and can´t progress, it can still run with 7 tps. Now with the Mtp (multitoken prediction) it nearly double the speed ( reached 82tps today with the 35b 100000ctx). I recommend it you give it a try.

trilogic 6mo ago

Really, how so? Because we work with codebases daily, can you tell us a concrete example! In our case we work in consumer hardware (ish), 10 million ctx (1 million output, 1 million input proven, sometimes it loops or breaks at over 500k ctx byt at ~17tps linear). IT can read the full codebase, unleash agents, and write in disk editing and patching files creating a full app in 3-4 minutes. IT can do Web search and Rag pretty fast, it understands and fix the user query, sys prompts and adapt/fix them if needed on the fly. I am wondering what more do you do?

trilogic 6mo ago
Edit: Forgot to mention that it can process images and pdf, and 100s of other files, it can even create presentations in code or mermaid, svg, charts js etc. Here a basic version of it: https://hugston.com/chat https://hugston.com/chat
- rspoerri 6mo ago
  how do you do 1mio context with qwen3.6 27b, that only supports 256k? and what hardware would you run that on? 2 * 3090 is afaik currently at max 256k context.
  - omneity 6mo ago
    You can increase the context window beyond its max trained context using RoPE scaling[0] which will require more VRAM. But you can increase your context window for the same VRAM by quantizing the KV cache with FP8 (double the context) or TurboQuant (more than double)[1]. 0: https://medium.com/@leannetan/extending-context-length-with-hugging-faces-transformers-6b04db05b39a https://medium.com/@leannetan/extending-context-length-with-... 1: https://docs.vllm.ai/en/latest/features/quantization/quantized_kvcache/#example-quantize-llama-attention-kv-cache-to-fp8 https://docs.vllm.ai/en/latest/features/quantization/quantiz...
  - trilogic 6mo ago
    We managed to increase the ctx for whatever llm model that is GGUFED, here the experimental tests: https://www.reddit.com/r/Hugston/ https://www.reddit.com/r/Hugston/
  - nyrikki 6mo ago
    You can get all the Qwen 3.x models up to ~1 million tokens using YaRN with llama.cpp.[0] Personally I am using `--no-context-shift` and feeding in context back in on failure at the harness level. I have 2x1080ti + 1xTitanV that have a full 262,144 tokens context on 262,144 tokens with `-sm tensor` at 62.04 t/s which isn't so bad. But I also have a 1x3090 running unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL at 41.89 t/s but with only 130k context, but if you have a modular programming style both work pretty well. But play with YaRN if you really need it. [0]https://qwen.readthedocs.io/en/v3.0/run_locally/llama.cpp.html https://qwen.readthedocs.io/en/v3.0/run_locally/llama.cpp.ht...

0xbadcafebee 6mo ago

> not for working on real codebases You don't pick just one model to "work on real codebases". You use a very advanced model to plan, and a not-very-advanced, cheaper, faster model to execute planned tasks. This saves money and speeds up work. This is the guidance from Anthropic & OpenAI.