5 ms·

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. Granite has recent training data which is nice. If the other small models got fine tuned on recent data I don't know if I would use this at all, but that alone makes it pretty decent. The 4b they released was not good for my needs but could probably handle tool calls or something

steveharing1 6mo ago
Yea, No doubt Qwen 3.6 open weights are far more strong
- rnadomvirlabe 6mo ago
  Why no doubt?
  - steveharing1 6mo ago
    Because Qwen 3.6 pushes way above its weight. Granite 8B is impressive, but Qwen still wins on raw capability, especially for coding.
    - actionfromafar 6mo ago
      Way above its weights.
      - drittich 6mo ago
        Nanobanana for scale.
    - rnadomvirlabe 6mo ago
      You just asserted the same thing again. Why do you say this is the case?
      - steveharing1 6mo ago
        [dead]
      - noodletheworld 6mo ago
        Having tried it. Qwen is really good. Also, generally, it makes sense. 8B models are generally not very good^. That this 8B model is decent is impressive, but that it could perform on par with a good model 4 times as large is a daydream. ^ - To be polite. The small models + tool use for coding agents are almost universally ass. Proof: my personal experience. Ive tried many of them.
        irishcoffee 6mo ago
        So it’s just like, your opinion, man? edit: It was a play on The Big Lebowski, folks.
        Terretta 6mo ago
        College SAT scores do not tell you how the dev applying for your open back end systems engineering job is going to do once they're in your workplace harness. Nor do class standings, nor hackerrank and the like. What will tell you is asking them to fix a thing in your codebase. Once you ask an LLM to do that, a dozen times, I'd argue it's no longer "just your opinion man", it's a context-engineered performance x applicability assessment. And it is very predictive. But it's also why someone doing well at job A isn't necessarily going to be great at B, or bad at A doesn't mean will necessarily be bad at B. I've often felt we should normalize a sort of mutual try-buy period where job-change seeker and company can spend a series of days without harming one's existing employment, to derisk the mutual learning. ESPECIALLY to derisk the career change for the applicant who only gets one timeline to manage, opposed to company that considers the applicant fungible. But back to the LLM, yeah, the only valid opinion on whether it works for you is not benchmark, it's an informed opinion from 'using it in anger'.
        robotmaxtron 6mo ago
        the (dead) internet is full of opinions exactly like this
    - locknitpicker 6mo ago
      [dead]
  - captainbland 6mo ago
    No comparison with competitor models other than the previous granite version strongly implies that it does not compete well with other comparable models. At least this is the most reasonable assumption until data comes out to the contrary
  - 2ndorderthought 6mo ago
    Qwen 36 is effectively a pocket sized frontier model. It's really surprising for me anyway
vessenes 6mo ago
Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.
- 2ndorderthought 6mo ago
  I tried the Gemma 4 I think 2 and 4b. The 2b was not useful for me at all. A little too weak for my use cases The 4b was okay. It didn't get all of my small math questions right, it didn't know about some of the libraries I use, but it was able to do some basic auto complete type stuff. For microscopic models I like the llama 3.2 3b more right now for what I do, it's a little faster and seems a little stronger for what I do. But everyone is different and I don't think I'll use it anymore this past month has been crazy for local model releases.
  - throwaw12 6mo ago
    can you share your use cases for 2b and 4b models? curious how people are leveraging these models
    - 2ndorderthought 6mo ago
      For me, I use them for quick auto complete or small questions. I am not a vibe/agentic coder. I know I am a relic and a Luddite because of this. Instead of hitting stack overflow and Google I will ask questions like "can you give me an example of how to do x in library y?" Or "this error is appearing what might be happening if I checked a b and c". Or "please write unit tests for this function". Or code auto complete. I am not looking for the world's best answer from a 3b model. I am looking for a super fast answer that reminds me of things I already know or maybe just maybe gives me a fast idea to stub something while I focus on something more important, I am going to refactor anyways. Think a low quality rubber duck I mostly use 7-9b models for this now but llama 3.2 3b is pretty decent for not hogging resources while say I have other compute heavy operations happening on a weak computer. Probably half the questions people ask chatgpt could get roughly the same quality of answer with a small model in my opinion. You can't fully trust an LLM anyways so the difference between 60% and 70% accuracy isn't as much are marketing makes it sound like. That said the quality of a good 7-9b model is worth it compared to a 3b if your machine can run it. Furthermore the quality of qwen 36 is crazy and makes me wonder if I will ever need an AI provider again if the trend continues.
cyanydeez 6mo ago
Qwen3-Coder-Next seems to be perfect sized for coding. I tried the new and just found the verbosity not really useful for coding. But probably for more analytical tasks or writing docs.
- UncleOxidant 6mo ago
  Qwen3-coder-next is still my favorite local model. Qwen3.6-27b is probably a bit better, but it also runs much slower on my Strix Halo box. Hoping we see a Qwen3.6-coder soon!
3abiton 6mo ago
> Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. I second this! Using the Unsloth Q6 (I forgot the exact name). Currently using it with forgecode (with zsh), on my Strix Halo, and it's suprisingly really good. I would say slightly Similar to Haiku 4.5, plus additional privacy, minus speed. It's surprisingly really fast for the hardware, given the speculative decoding, still PP is on the slow side.
- lostmsu 6mo ago
  If you use it for agentic coding and often hit PP, there's something wrong with your harness IMO
- bpye 6mo ago
  Out of interest, what are you seeing for token generation - especially as the context fills?
- jatora 6mo ago
  curious, how is forgecode? have you compared it to codex/opencode/claude code?
UncleOxidant 6mo ago
> I may use this for auto complete Using an 8B LLM for auto complete seems kind of like overkill. Couldn't a much smaller model handle that? IIRC there's a Qwen 1B model.

steveharing1 6mo ago

Yea, No doubt Qwen 3.6 open weights are far more strong

rnadomvirlabe 6mo ago
Why no doubt?
- steveharing1 6mo ago
  Because Qwen 3.6 pushes way above its weight. Granite 8B is impressive, but Qwen still wins on raw capability, especially for coding.
  - actionfromafar 6mo ago
    Way above its weights.
    - drittich 6mo ago
      Nanobanana for scale.
  - rnadomvirlabe 6mo ago
    You just asserted the same thing again. Why do you say this is the case?
    - steveharing1 6mo ago
      [dead]
    - noodletheworld 6mo ago
      Having tried it. Qwen is really good. Also, generally, it makes sense. 8B models are generally not very good^. That this 8B model is decent is impressive, but that it could perform on par with a good model 4 times as large is a daydream. ^ - To be polite. The small models + tool use for coding agents are almost universally ass. Proof: my personal experience. Ive tried many of them.
      - irishcoffee 6mo ago
        So it’s just like, your opinion, man? edit: It was a play on The Big Lebowski, folks.
        Terretta 6mo ago
        College SAT scores do not tell you how the dev applying for your open back end systems engineering job is going to do once they're in your workplace harness. Nor do class standings, nor hackerrank and the like. What will tell you is asking them to fix a thing in your codebase. Once you ask an LLM to do that, a dozen times, I'd argue it's no longer "just your opinion man", it's a context-engineered performance x applicability assessment. And it is very predictive. But it's also why someone doing well at job A isn't necessarily going to be great at B, or bad at A doesn't mean will necessarily be bad at B. I've often felt we should normalize a sort of mutual try-buy period where job-change seeker and company can spend a series of days without harming one's existing employment, to derisk the mutual learning. ESPECIALLY to derisk the career change for the applicant who only gets one timeline to manage, opposed to company that considers the applicant fungible. But back to the LLM, yeah, the only valid opinion on whether it works for you is not benchmark, it's an informed opinion from 'using it in anger'.
        robotmaxtron 6mo ago
        the (dead) internet is full of opinions exactly like this
  - locknitpicker 6mo ago
    [dead]
- captainbland 6mo ago
  No comparison with competitor models other than the previous granite version strongly implies that it does not compete well with other comparable models. At least this is the most reasonable assumption until data comes out to the contrary
- 2ndorderthought 6mo ago
  Qwen 36 is effectively a pocket sized frontier model. It's really surprising for me anyway

vessenes 6mo ago

Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.

2ndorderthought 6mo ago
I tried the Gemma 4 I think 2 and 4b. The 2b was not useful for me at all. A little too weak for my use cases The 4b was okay. It didn't get all of my small math questions right, it didn't know about some of the libraries I use, but it was able to do some basic auto complete type stuff. For microscopic models I like the llama 3.2 3b more right now for what I do, it's a little faster and seems a little stronger for what I do. But everyone is different and I don't think I'll use it anymore this past month has been crazy for local model releases.
- throwaw12 6mo ago
  can you share your use cases for 2b and 4b models? curious how people are leveraging these models
  - 2ndorderthought 6mo ago
    For me, I use them for quick auto complete or small questions. I am not a vibe/agentic coder. I know I am a relic and a Luddite because of this. Instead of hitting stack overflow and Google I will ask questions like "can you give me an example of how to do x in library y?" Or "this error is appearing what might be happening if I checked a b and c". Or "please write unit tests for this function". Or code auto complete. I am not looking for the world's best answer from a 3b model. I am looking for a super fast answer that reminds me of things I already know or maybe just maybe gives me a fast idea to stub something while I focus on something more important, I am going to refactor anyways. Think a low quality rubber duck I mostly use 7-9b models for this now but llama 3.2 3b is pretty decent for not hogging resources while say I have other compute heavy operations happening on a weak computer. Probably half the questions people ask chatgpt could get roughly the same quality of answer with a small model in my opinion. You can't fully trust an LLM anyways so the difference between 60% and 70% accuracy isn't as much are marketing makes it sound like. That said the quality of a good 7-9b model is worth it compared to a 3b if your machine can run it. Furthermore the quality of qwen 36 is crazy and makes me wonder if I will ever need an AI provider again if the trend continues.

cyanydeez 6mo ago

Qwen3-Coder-Next seems to be perfect sized for coding. I tried the new and just found the verbosity not really useful for coding. But probably for more analytical tasks or writing docs.

UncleOxidant 6mo ago
Qwen3-coder-next is still my favorite local model. Qwen3.6-27b is probably a bit better, but it also runs much slower on my Strix Halo box. Hoping we see a Qwen3.6-coder soon!

3abiton 6mo ago

> Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. I second this! Using the Unsloth Q6 (I forgot the exact name). Currently using it with forgecode (with zsh), on my Strix Halo, and it's suprisingly really good. I would say slightly Similar to Haiku 4.5, plus additional privacy, minus speed. It's surprisingly really fast for the hardware, given the speculative decoding, still PP is on the slow side.

lostmsu 6mo ago
If you use it for agentic coding and often hit PP, there's something wrong with your harness IMO
bpye 6mo ago
Out of interest, what are you seeing for token generation - especially as the context fills?
jatora 6mo ago
curious, how is forgecode? have you compared it to codex/opencode/claude code?

UncleOxidant 6mo ago

> I may use this for auto complete Using an 8B LLM for auto complete seems kind of like overkill. Couldn't a much smaller model handle that? IIRC there's a Qwen 1B model.