11 ms·

Qwen 3.8 27B is excellent, but it defaults to overthinking things

• andy99 1mo ago
  The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens. I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point that the absolute tok / s didn’t really matter.
  • simonw 1mo ago
    Comparing with Muse Glimmer is a good idea. I ran the same exact HTML tool generating prompt against both Glimmer 30B and Qwen 3.8 27B. Results: Qwen: https://gist.github.com/simonw/121ad098860028b2fab603fa12da1fd9 https://gist.github.com/simonw/121ad098860028b2fab603fa12da1... - 17,576 reasoning tokens, produced this HTML result: https://static.simonwillison.net/static/2026/qwen-over-thinking-bbox.html https://static.simonwillison.net/static/2026/qwen-over-think... Glimmer: https://gist.github.com/simonw/51e8ddb2ee597a5005fa63bd4927dbf0 https://gist.github.com/simonw/51e8ddb2ee597a5005fa63bd4927d... 1,021 reasoning tokens, this HTML: https://static.simonwillison.net/static/2026/glimmer-bbox.html https://static.simonwillison.net/static/2026/glimmer-bbox.ht... - ugly but functional. In both cases paste in the URL https://static.simonwillison.net/static/2026/two-pelicans-on-rocks.jpg https://static.simonwillison.net/static/2026/two-pelicans-on... to see them work. Both applications work correctly and fulfill the requirements. The Qwen one (which used the default xhigh reasoning setting) is massively over-engineered. The Glimmer one used whatever their default in LM Studio is and I would argue is a tiny bit under-engineered. Weirdly the Glimmer one doesn't work with images on other domains like https://static.inaturalist.org/photos/714731804/large.jpg https://static.inaturalist.org/photos/714731804/large.jpg - it fails with a CORS error, but you don't need CORS to load images and detect their width and height, and the Qwen one handles that URL just fine. That's because Glimmer added this unnecessary line: img.crossOrigin = 'anonymous';
    • NitpickLawyer 1mo ago
      Yesterday I tried both as well. I do a quick "explain this repo" + "any security issues" convo to do a "vibe check" on the models and make sure everything works w/ serving and harness. Both qwen and glimmer explained it pretty well, and both accepted the security question without any issues, flagged a few things left there on purpose (hardcoded tokens, single auth, no logs, etc). I like the style of glimmer more. Much terser language, no adjectives, no fluffy claude-like language. ("Images are written to...", "Tasks are stored in SQLite...", "Docker image is built from ...") In contrast, qwen is a bit more flowery. ("Unbounded image processing / resource exhaustion — preprocess() opens whatever was downloaded with no size/dimension/format validation before the VAE encodes it..." , "SQLite as a queue — fine at this scale, but...", "Debug info leakage — exceptions are re-raised as...". But both flagged pretty much the same stuff, just ordered / styled differently. Mighty impressive understanding for a thing that I can run locally. Qwen served in fp8 w/ full kv cache, glimmer in w4a16 (the fp8 weights wouldn't serve for whatever reason), both at full supported context in 48GB of VRAM.
  • bogzz 1mo ago
    I love reading Glimmer's "thoughts". Why use many word when few do trick?
    • dofm 1mo ago
      It’s also a little bit snarky, almost. The stuff it thought during the car wash puzzle made me laugh.
      • shifto 1mo ago
        It wouldn't accept it was the Glimmer model and I had to stop when I realized I was arguing with my GPU. Was a surreal moment, lol.
    • Gracana 1mo ago
      I’ve noticed dsv4 do that as well, but inconsistently. I thought it was broken at first, but no, it’s just kind of shorthand that it does while thinking.
    • bblb 1mo ago
      Me machine, no human. Why waste token. Do fast, deliver.
  • lostmsu 1mo ago
    Glimmer is stupider than 3.6 27B. You can't compare its speed to 3.8 and be done.
• kamranjon 1mo ago
  A no-thinking pelican! I hope to see more, it's surprisingly good for just 2 minutes.
• deadcatfound 1mo ago
  [flagged]
• SwellJoe 1mo ago
  This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately (https://github.com/swelljoe/flar/pull/17 https://github.com/swelljoe/flar/pull/17), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did a similar task in about 20 minutes. Most big models took about an hour or so, and most small models needed a couple of hours (but did a worse job).
  • simonw 1mo ago
    Was that with the default xhigh reasoning setting? I suggest trying again with reasoning set to low or turned off entirely.
    • SwellJoe 1mo ago
      Yes, default everything, no tuning, 8_K_XL Unsloth quantization on dual Radeon V620 GPUs (which aren't blazing, but faster than the Strix Halo).
      • syntaxing 1mo ago
        What speed do you get on this setup? Im tempted to use the same GPU.
        • SwellJoe 1mo ago
          It varies widely based on a bunch of factors. With this specific model at 8-bit quantization and MTP, it starts out at about 25 t/s for basic chat, but for agentic tasks with long context it slows way down to something like 12-15. I don't see a big difference in token rates based on any config changes I have tried, or going to the smaller 6-bit quantization, so far, though I haven't spent a ton of time on experimenting. If you already have one or more of them, then, yeah, you can use them for this model or any other at around this size or smaller, but I wouldn't recommend you buy them (or anything else, right now, everything is just too overpriced). You can run better models for less money at higher speeds. I bought mine before they got more expensive, but I wish I'd just bit the bullet and bought newer/faster cards before they got more overpriced. Or, the actual smart money, even back then was to just use cloud models and forget about self-hosting.
• javchz 1mo ago
  I wonder if this can be fixed with LORAs.
  • bitexploder 1mo ago
    I had to fix this on 35B A3B -- I have a proxy that just shuts it down if it gets to 2K thinking tokens and injects something like "We have thought enough, let's begin working." and it almost always finishes the turn then. It rarely needs more than 2K thinking tokens and if it does there is always next turn. I would need to see what 27B is actually doing, but these smaller Qwen models seem prone to this.
    • dofm 1mo ago
      Unfortunately in xhigh reasoning effort it will burn through 2K tokens before it has even finished its bullet point overview. It really is intense and obsessive. You might need ten times more! Your strategy would likely help in medium reasoning effort (because there it gets caught up in the very typical Qwen looping). Not seen looping in the “low” reasoning effort mode.
      • bitexploder 1mo ago
        I have been using Muse Glimmer for a few days instead of A3B. It gets the job done quicker than A3B despite being several times slower.
        • dofm 1mo ago
          Yes — I just found out that you can set reasoning level in the prompt — like with Qwen 3.8 27B it is actually really pretty solid at "Reasoning level: low". Ten to thirteen tokens per second on my M1 Max (might be some room to improve this) but it indeed solved as fast as the Qwen 35B. 40 seconds faster on one of my tests that involves three steps. This is very striking.
    • logicallee 1mo ago
      >I had to fix this on 35B A3B -- I have a proxy that just shuts it down if it gets to 2K thinking tokens and injects something like "We have thought enough, let's begin working." and it almost always finishes the turn then. that is amazing, thanks for sharing.
• cyanydeez 1mo ago
  --thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing. the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do. If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion of the thinking traces, and redirecting it. I use the message to tell it to use subagents, add additional logging and to use opencode's dynamic context pruning. As such, we'll just whisper here _skill issue_.
  • dofm 1mo ago
    Unfortunately in xhigh thinking it goes down rabbit holes in such an extreme depth-first way, that whenever you choose to cut it off, there is a very good chance it will not have got round to musing on even half of the prompt! It doesn’t really obviously loop in xhigh, so I am not sure if an “overthinking guard” proxy would have much to go on, but it does obsessively ruminate on edge cases. I have seen it overcomplicate simple code as a result even in my limited testing. Probably the better solution if you want it to be quicker but still fairly thorough appears to be to configure reasoning effort instead of thinking budget. It seems to do very well still even on the Low setting; on the Medium setting it can get stuck in loops like 3.6 does. I think xhigh reasoning effort was an absurd choice for a default, and so was not sorting out the chat template so LM Studio could offer the reasoning effort dropdown.
    • cyanydeez 1mo ago
      to the point though: most of that overthinking is useless if you have a proper redirect message. So setting arbitrary budget and getting it a good message will do the trick regardless of what type of thinking it's doing. The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it. The only think I could think that'd be better than the --reasoning-budget would bet a budget jitter just in case it really is repeating a pattern and you want to escape it arbitrarily, otherwise yes, it could keep looping if you're always cutting at the wrong time.
• deleted 1mo ago
  [deleted]
• LoganDark 1mo ago
  I hope Apple does end up moving to HBM. Unified memory has been a huge godsend, but the low memory bandwidth is just such a killer. Even/especially on M5, where the available compute is starting to starve incredibly badly on ML workloads.
  • dofm 1mo ago
    AFAIK that is initially only for the iPhone?
    • LoganDark 1mo ago
      Apple is reportedly considering skipping the higher-end M6 chips altogether, which could potentially give enough time for higher-end M7 (in over a year) to use HBM
      • kennywinker 1mo ago
        Considering, or being forced to consider due to the hardware crunch?
        • LoganDark 1mo ago
          Who cares? If high-end M7 chips come out with HBM and actually compete with GPUs from this decade I'll be really happy. I'm also hoping for more than 128GB of unified memory in a laptop.
• xscott 1mo ago
  It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose: User: We're going to make <silly demo>. Please create a plan, but do not write code yet. Agent: <short and reasonable plan> User: Now please follow that plan and write the code. No other chat. Agent: <reasonable code in reasonable time> Maybe this can be fixed with Jinja templates or something, or maybe it's a hack to your harness, but it shows you can get the model to reason reasonably.
  • adam_arthur 1mo ago
    Yes, if you set reasoning to none you can force the granularity of the thinking. It will actually adhere to your request for e.g. 3 sentences max. Thinking mode will override any instructions in the prompt (at least for other models in my experience). Of course this will probably hurt performance, but works great for easy tasks that you know are trivial. Tons of pipeline, image recognition etc use cases where this works well. I'd be curious to see Qwen 3.8 27B low thinking benchmarks though.
  • hedgehog 1mo ago
    To be fair a lot of models have quirks, I've never found a model swap that was transparent.
  • regexorcist 1mo ago
    I'm doing much the same, avoid the long thinking loops and instead have more iterations on the plan with reviews from different angles.
  • theshrike79 1mo ago
    I feel that local models are better for "processes" where you need a degree of predictability. Like summarising the daily weather for the family chat bot or analysing email inbox priority. SOTA cloud models are more for open-ended tasks where you need "creativity".
  • vanviegen 1mo ago
• blagui 1mo ago
  You have 4 thinking levels. You can disable it. It's well known issue in Qwen, previous releases I would disable it by default. Also xhigh seem a new thing.
  • dofm 1mo ago
    Yes. Though the chat template doesn’t tell LM Studio to offer the little dropdown. You can bodge the template in the load parameters. Unsloth Studio / Desktop has it working really well with their version of the weights.
  • Anonyneko 1mo ago
    So it can be disabled on 27B after all? I haven't downloaded 3.8 27B yet, but the announcement made it seem like only the hosted Max can disable thinking, and with the open weights you could only select between provided reasoning efforts.
    • simonw 1mo ago
      I've been passing "reasoning": { "effort": "none" } to LM Studio's Responses API and the resulting response doesn't include any reasoning chunks, so that seems to work.
• doginasuit 1mo ago
  To be fair, Opus 5 overthinks things on a regular basis. I interact with the LLM almost entirely through the prompt interface vs. some agentic harness, so I have a lot of granular exposure to its reasoning. For almost every code analysis, it flags all the important issues and at least one non-issue. It suggests some impractical and unnecessary fix for the non-issue that would categorically be a regression. I've learned that medium effort can improve the outcome relative to higher settings. But I suspect the phenomenon is an artifact of a misguided effort to fix inherent LLM limitations. At least some of its reasoning will miss the target, and more bad reasoning is not the remedy.
• nharziro 1mo ago
  I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details: https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1bf251d https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b...
  • matheusmoreira 1mo ago
    Opus 4.6 performance with a local model that can be hosted on consumer hardware is an incredible result!!
    • nharziro 1mo ago
      I was genuinely surprised because it's quite a leap from where 3.6 was an as far as I understand this isn't a new model, it's the same model that's been post trained, so I don't quite understand what they did to improve it so substantially. The previous model couldnt get through the benchmark at all. Though it remains terribly inefficient and slow. The hardware will have to get substantially faster for these kinds of models to be daily drivers. I think I forgot to mention in the bench that I ran it on an m5 max mac book
• RachelF 1mo ago
  To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago. I hope this trend continues.
  • refactor_master 1mo ago
    Unlike cloud infra in general which offers things like automatic backups, regional redundancy, and effectively unlimited scalability, it seems like the value proposition of cloud LLM gets ever shakier. * Many businesses don't need frontier level intelligence anyway. * It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.
    • manmal 1mo ago
      Yeah. Local agent sessions are not backed up in the cloud. And uptime is better with local models.
    • SV_BubbleTime 1mo ago
      [dead]
    • NhanH 1mo ago
      The whole cloud story lies on two aspects: - Hyperscaling “we are going to serve billions of people in our applications”, which is becoming increasing unlikely as regional tech companies become more dominant than than the global one (this one is as much about geopolitics as technology) - Operations is hard, in which case non-frontier models should be increasingly capable. Devops for small-ish deployment is one of the few cases where it is hard to clam you need deep expertise and AI can’t do it. Previously, the claim is that you need people specialized in ops, which is expensive. Now… My prediction is that not just cloud LLM, but cloud business general will have to change. Not yet in the next 5 years, but probably 8-20 years-ish
      • redrove 1mo ago
        I don’t disagree but saying “cloud will change in the next 8-20y-ish” is a bit of a non-argument, you’re not really stating any thesis to speak of; Change is a given over that time frame.
• xlayn 1mo ago
  I have this branch of llama.cpp that among other things (like patching the template to not break the kv cache, and saving conversations to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-eviction https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... I did testing and the reasoning effort can be set per message, I was not aware of the option of none mentioned by @xscott, I tested but didn't see any change, I think there are just 3 values, xhigh, medium and low as per https://huggingface.co/Qwen/Qwen3.8-27B-FP8 https://huggingface.co/Qwen/Qwen3.8-27B-FP8 , I did testing and the thing can do it's "I'll speak 10 million words to myself to ensure I'm not missing something" and then switch to a faster model, then switch... I did a test and the thing keep coherence and follow it's train of though-kens, you can see the result here... https://github.com/alainnothere/llama.cpp/blob/disk-cache-eviction/effort_alternation_test/results/run-2608141400-qwen38-64kctx.txt https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...
  • Gracana 1mo ago
    What’s that about the template breaking the kv cache?
    • xlayn 1mo ago
      this is my understanding, the default template keeps the thinking part but only for the last message, so the harness has to play along with the template and strip and add to keep the conversation matching what's there on the llama.cpp cache, but if the harness sends the thinking in every turn, then you break what llama.cpp expects, the conversation doesn't match anymore what you have on cache and it reprocesses again the whole conversation
      • Gracana 1mo ago
        Oh, interesting. I want to say there’s a “preserve thinking” option in mainline llama.cpp, but I’m seeing mixed results in my searches and my in-flight internet is not good enough to dig into it properly. Good to hear you have a solution though, because that is likely something I’ll run into as well.
• deleted 1mo ago
  [deleted]
• jatora 1mo ago
  All current era models overthink as it's a product of their RL incentives (or distillation of models with them...) From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like: Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively. That is fantastic for SWE benchmarks and autonomous agents. It also naturally creates pathologies: under-answering is expensive; over-answering is cheap.
  • nojs 1mo ago
    For a model this small it’s also a way of trading supply-constrained VRAM for inference time, which for self hosting consumers (and probably Chinese companies subject to export controls) is mostly good. I can imagine a curve where for a given level of “intelligence” you either need model size, or inference time (“test time compute”), and can somewhat trade one for the other.
    • icelancer 1mo ago
      > I can imagine a curve where for a given level of “intelligence” you either need model size, or inference time (“test time compute”), and can somewhat trade one for the other. This is exactly what was shown on Luna/Terra/Sol tradeoffs - Luna requires much higher reasoning efforts to approach Terra/Sol on lower reasoning. Which is fine, of course, no complaints - but true.
  • dannyw 1mo ago
    The recent 'Stolen Thoughts'[1] paper shows many excerpts of private reasoning for frontier models. For a complex maths problem, Sol reasoned in 367 tokens before working: We need solve. Need interpret no person sits next to two other people = among binary string length16 weight8, no occupied chair whose both neighbors occupied, equivalently ab 111 substring. Endpoints cannot have two neighbors anyway; only avoid 111. Count binary strings length16 weight8 avoiding 111. Need N mod1000. Compute stepwise perhaps runs of 1 length max2. Count via runs. [... cut in half for HN readability ... ] Check interpretation potentially "no person sits next to two other people": no seated person's chair adjacent to two occupied chairs. Exactly no three consecutive chairs selected. yes. Need reason step by step final boxed 907. Explain runs. Ensure people each select chair distinct subset (given subset count). Let's present. That doesn't look like an overthinker to me, and matches my experiences. There's plenty of papers and research on reducing thinking verbosity/length while keeping as much quality as possible. I think one of the bigger problems is that verbose, `max`-style thinking does generally lead to higher benchmark scores. And model vendors are incentivised to for benchmarks (at least to some extent). [1] https://stolen-thoughts.com/ https://stolen-thoughts.com/
• onSmallMessage 1mo ago
  [dead]
• jakswa 1mo ago
  I went back to Glimmer 30b for my 20GB of VRAM. Just a better experience fit-wise and speed-wise and tone-/voice-wise.
• johnnyApplePRNG 1mo ago
  According to the paper "Stealing reasoning traces from proprietary llms" [0] all frontier models overthink. Thinking is good. You just don't see it in proprietary harnesses because it's literally cryptographically hidden from you. [0] https://arxiv.org/pdf/2608.09867 https://arxiv.org/pdf/2608.09867
• pranav_tech26 1mo ago
  [flagged]
• jedbrooke 1mo ago
  I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought
  • russfink 1mo ago
    It “thinks out loud” to populate its token space. Asking it to shortcut risks truncating that process.
    • jauntywundrkind 1mo ago
      Yeah. It's "thinking" in absurd massive vectors. It needs to assess a couple to weigh out. That's the compression. That's the nature. It looks ridiculous when thinking traces render out such simple statements ('reassessing ..') but I expect this is far deeper an assessment than it can fully reflect to us on, and I expect its a huge part of their thinking.
  • NewJazz 1mo ago
    Hardcode their "thoughts" in your agents.md... But they might still reason through it anyway.
  • suprjami 1mo ago
    So-called "caveman" thinking attempts to address this. The important part of "actually wait, I really need to XYZ" is just "XYZ". The model can attend to just "do XYZ" and produce almost the same vector modifications as full verbose "reasoning".
    • vanviegen 1mo ago
      I don't think that's true. If a context contains a statement followed by something opposing that statement, that will confuse the model. So "actually wait, I really need to" is there to signal that the previous thinking may be flawed and that what follows is a new attempt. It's good to remember that LLMs have no more state then what they can derive from the context up til any point. So if that context is hard to interpret, that will reduce effectiveness.
• matheusmoreira 1mo ago
  Am I the only one who enjoys it when LLMs overthink everything? Opus 4.8 would spend like 10 minutes thinking and then go out there and do an excellent job. Only Fable 5 seems to be smart enough to just know everything it needs to immediately start working without any reasoning or verification. Opus 5 tries to be relentless like Fable, but it's not as smart as Fable and I have to constantly challenge and correct its unfounded assumptions. Sol is somewhere between Fable and Opus 5, it's smart but it's not Fable, it keeps making assumptions that I have to correct. After trying all these models, I find that I miss Opus 4.8's overthinking. Sure it's slow, but it actually gets things right.
  • jongjong 1mo ago
    Yes, for coding, they aren't overthinking enough. I want much more thinking and less code in the PR! Even with the best frontier models, I still have to guide them towards the right solution. The more thinking they do, the less code they write. I have quite a complex codebase where I made a lot of nuanced decisions with regards to keeping the processes embarrassingly parallel, DB indexing, caching, async/await, backpressure monitoring, spam prevention, schema validation, etc... and now the agents are really good at adding features on top and prompting is minimal. If you have a relatively large codebase and never even once cut a corner, then the AI agents tend to follow through with that style and the ratio of reasoning-to-code increases. Worth it.
  • kzrdude 1mo ago
    Depends on how you work with it. Reading the meandering and repetitive thinking is disturbing and taxing, so we can’t do that. So thinking has to be (mostly) hidden and just becomes waiting time.
• teravor 1mo ago
  when you distill a thinking LLM past its capacity it will default to overthinking because during training that was the only way for a chance at a reward on many tasks. you can generally avoid this if you specialize it on a domain that is within its capacity.
  • mordae 1mo ago
    I think that in this case there is also the problem of trying to transfer MoE-style reasoning into a dense model. I mean, MoE needs reasoning to walk multiple experts, but dense model already has all the weights. So when you push it hard to reproduce the MoE traces, you are effectively asking a small mouse to role-play as an anthill. Not great.
• PrimeAli 1mo ago
  [flagged]
• chvid 1mo ago
  “The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
  • AgentMasterRace 1mo ago
    his 128gb Ram laptop is quite extreme
    • simonw 1mo ago
      It should just about be usable in 32GB.
      • npodbielski 1mo ago
        It is. I am running it on R9700
      • krzyk 1mo ago
        On a consumer hardware it would be nicer. With no GPU/iGPU or a 6-8GB VRAM.
        • hnfong 1mo ago
          It would be somewhat slow on a CPU only machine, but it still works. Besides, Macbooks with 32GB RAM is consumer hardware, just maybe on the higher end.
          • krzyk 1mo ago
            Well maybe, but a non-macos laptop is a bit more common.
      • bakraman 1mo ago
        RAM is never the issue, it's always the compute power
• atif089 1mo ago
  So if I have to set this up on my 24GB MBP what'd the right configuration and tuning look like?
  • suoloordi 1mo ago
    Checkout MTPLX. I use the Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed model and I get around 12 tok/s on a 36GB MBP.
    • dofm 1mo ago
      I still haven't tested MTPLX and the memory of reading about it had drifted out of my brain like so many things have, so thanks! I really must.
• TechSquidTV 1mo ago
  Ironically I had just installed omlx, tried 3.8 27b 8bit and then Googled about it overthinking, then this was the first result. 4 hours old.
  • reilly3000 1mo ago
    The feedback loops are getting tighter every day.
• semiinfinitely 1mo ago
  some people just dont understand the concept of a leaked benchmark
  • simonw 1mo ago
    You mean this? draw an svg of a circle
    • semiinfinitely 1mo ago
      yeah exactly
• ComputerGuru 1mo ago
  Complaining about overthinking in xhigh then pointing out output had bugs with thinking turned off seems like it’s missing the obvious compromise?
  • icelancer 1mo ago
    I think the point is that a larger model with far less reasoning time solves the problem just fine. Which of course is the tradeoff: The smaller the model, the more reasoning you need to get decent answers to tough questions.
  • solarkraft 1mo ago
    Agreed. Test driving the bad default is what they deserve (they brought this upon themselves as a benchmaxx attempt), but comparing it to running with reasoning completely disabled is also weird.
• npodbielski 1mo ago
  On the other hand I am running this model to write some tests for my hobby project for two days now and it is able to deduce and fix errors and bugs that Qwen 3.6 was not able to. Yes, it thinks a lot but this makes reasoning about problem much better. Also it did not run it self into a loop once even which is a problem with Q4 even with dense models. On the other hand it maybe do too much i.e. I asked "how we could test it?" and instead of answering it just actually wrote tests. But it was the same with Qwen 3.6.
• chaostheory 1mo ago
  I prefer that to the under thinking that both Gemini and the newer Grok models do
• dehrmann 1mo ago
  Clicking through is worth it just for the "draw an svg of a circle" bit.
• elisbce 1mo ago
  I tried it and it performed poorly on my private benchmark problems. The overthinking problem is real, it takes 5-10x the reasoning tokens than comparable models. It is a sign of inadequate training of the base model and it is using more reasoning tokens to compensate for that. I also noticed that it is likely to get into somewhat repetitive reasoning and forgetting about some user requirements, suggesting that it could be the side effects of using 3:1 linear attention vs full attention.
• m3kw9 1mo ago
  This could turn nasdaq red tmr
• CodeWithLeo 1mo ago
  [flagged]
• solarkraft 1mo ago
  I find that a lot of the recent allegedly great open models are cranking their reasoning way further than I find reasonable for interactive use. I’m writing this while waiting for the new Deepseek V4 Flash to finish its task, which is taking way longer than the older version. What gets reported is always the benchmark result, but rarely the real-world trade-off made to achieve it. That’s an obvious incentive for the labs, so I think Simon is correctly zeroing in on it. Please continue doing so for models that don’t go too far as much as this release. Don’t get me wrong, I think it’s amazing what we can get out of smaller models with more reasoning, but we should be super aware how very much not-free it is. This is a good opportunity to call out models that reason quickly: Meta’s Glimmer seems to be pretty token efficient so far, as do the GPT 5.6s.
  • dannyw 1mo ago
    The model itself is excellent, the defaults are bad. As Simon and other pointed out, medium is great. Reminds me of Gemma4 and the official (or at least popularly used around launch) Jinja templates being wrong and broken for tool calling.
    • qlte 1mo ago
      Seems less like "bad"/broken defaults and more defaults tuned to the max for benchmarks. All the positive PR from "Opus 4.6 level" online buzz is well worth the minor annoyance from taking a half hour to solve a simple problem since a user just needs to turn down the reasoning knob if it bothers them.
      • solarkraft 1mo ago
        It’s both: The default is bad, but not by accident. Since they certainly chose this default intentionally to be evaluated by it, they entirely brought it upon themselves for it to be evaluated as slow, overthinking and overcomplicating things. It’s a similar level of dishonesty as trying to conflate “starts at” vs. “tested configuration” car prices. We change the incentive to do this by evaluating it exactly as advertised.
  • 1mo ago
• madhu_ghalame 1mo ago
  [dead]
• chrismsimpson 1mo ago
  Surely this is great for an end user: the taste as to “when” and to what degree a model should “think” is now entirely in the fine tuners hands
• fzero 1mo ago
  This evolution of models that doesn't only favours big US corps is just good for humanity
• monksy 1mo ago
  I'll have to post the links to my Pelican svg. I did it in Q8 and BF16. The Q8 turned out better. But what I did see is that it does overthink a lot. 17GB is Q4 for Qwen3.8. That's quantitized quite a bit.
• dexterlagan 1mo ago
  I run mine on an M5 Max with just 48GB of (V)RAM, and it fits nearly twice in Q4. Works perfectly. I'm kinda glad I didn't spend the extra $2400 to get 128. We don't really need more... and that's a good thing (tm). God knows I thought about it in store. But I thought... maybe this year will be the year of the local model? Maybe soon we won't need that much RAM? I was right. The fact that it runs at 15tk/s in power saving mode, and 30 in perf. mode blows my mind. I can run the model in the background, coding something for me in OpenCode, hosted in LMStudio, while doing something else. What a world we live in. Having something close to human intelligence (at least for reasoning and code), running on a laptop, is amazing.
  • sgt 1mo ago
    For day to day LLM experimentation (and even some business use cases), I'd say Apple Silicon would be first choice for me.
  • vorticalbox 1mo ago
    Have you looked at using oMLX? https://omlx.ai/ https://omlx.ai/
    • mc8098 1mo ago
      Would second this, I switched to oMLX I get ~75 tok/s on Qwen3.6-35B-A3B-4bit on a 48GB M5 Pro
• XCSme 1mo ago
  My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh. Low and medium are basically the same. Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding. [0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen3-8-27b-medium/qwen-qwen3-8-27b-low/qwen-qwen3-8-27b-none/ https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...
  • monster_truck 1mo ago
    $0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here! If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo
    • XCSme 1mo ago
      I removed the extra links to sources for the electricity prices, but that's the average cost in EU, where I live. https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20260505-1 https://ec.europa.eu/eurostat/web/products-eurostat-news/w/d...
      • SimplyUnknown 1mo ago
        Indeed, EPEX spot prices are between €0.30 and €0.39 today including taxes where I live. I've seen them go up to €1.10 during peak demand.
      • hokkos 1mo ago
        it is not the marginal price for household users, but the (average electricity bill : subscription + consumption) / (average consumption), so it will be higher than the additional kwh consumed.
        • XCSme 1mo ago
• pranav_tech26 1mo ago
  [flagged]
• hellajack3d 1mo ago
  I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly. Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/master https://github.com/laurencehardman/llama-mindcontrol/tree/ma... Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful.
  • DarmokJalad1701 1mo ago
    Is that similar to what ggerganov is talking about here? https://x.com/ggerganov/status/2089214161884414147 https://x.com/ggerganov/status/2089214161884414147
    • hellajack3d 1mo ago
      Yes - it would seem so :) I did make a PR to the official llama-cpp repo some time back (about a month or so), but abandoned it as there seemed to be too much community concern that the mechanism would degrade model performance... Perhaps i'll polish it up and put some effort into benchmarking and revive the project in the near future.
      • mobelkh 1mo ago
        would be great as an opt-in though
  • kroaton 1mo ago
    This will degrade performance significantly. LLama.cpp has had this for a while and it tanks benchmark performance. I ran GPQA on GLM 5.2 using the llama implementation and it came back 19 points under the regular results.
• Balinares 1mo ago
  Worth noting that the default GGUF template sets the reasoning to xhigh. You can use the Froggeric template to set reasoning to medium instead: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates Worth noting as well that the weights come with an MTP layer that seems particularly accurate, to the point it can give you up to 6-8 correctly predicted tokens consistently enough to be useful. Which obviously boosts its speed enormously. I find it difficult to believe how good this model is. It feels like it's lagging heavyweight frontier models by less than a year, and it runs on your PC.
• ramon156 1mo ago
  How come the result didn't mention any timings? That's the one thing I was curious about. TTFT is quite slow on my machine because I do not have a GPU on hand right now (e.g. qwen3 coder was 8min)
• romeinaday 1mo ago
  Can you run this on a 36GB MacBook Pro (M3 Pro)? What would be a good setup? for coding mainly
  • petu 1mo ago
    You need ~24-26GB for basic setup (17-19GB model + 128K 8 bit context), so you can, but not much memory would be left for doing anything else on that machine. And even then it would run at like 5-10 t/s (due to relatively low memory bandwidth of M3 Pro) and slow prompt processing (couple hundreds of t/s?) If they end up releasing updated 35B-A3B variant, then it would be much more interesting in generation speed (~50 t/s) For inference engine/server you have two (main) choices: llama.cpp for platform-agnostic, MLX for Apple-only. They will spin up OpenAI-compatible local server, and you point your agent harness to it. For llama.cpp this should be reasonable (maybe shrink context to 128K) starting point: https://x.com/ggerganov/status/2088312671196082312 https://x.com/ggerganov/status/2088312671196082312
    • romeinaday 1mo ago
      So you can use llama.cpp and point a harness to it? What harness do you use?
  • akg_67 1mo ago
    [dead]
  • dstryr 1mo ago
    i have this exact same machine and 3.8-27b runs well, albeit slow at about 6 t/s. i have it complete tasks in the background as i work on other things. for comparison, my 3090 runs it at about 60 t/s.
    • romeinaday 1mo ago
      What are you using to run the model and as coding harness?
      • 1mo ago
• mdp2021 1mo ago
  Don't we have benchmarks for thinking quality - assessment over the "reasoning" output (correctness, structure, efficiency...)? We definitely should. And before the benchmark of the finished LLM, it would be interesting to consider the techniques used by LLM producers during training to optimize the "think" chunk quality. I cannot remember any good articles about it now.
• Balinares 1mo ago
  Reproducing the post's money quote here because it's absolutely the crux of why the Qwen 3.8 release is seismic IMO: "The models at this size continue to get better at an impressive rate. We don’t need to spend half a million dollars on datacenter-class hardware just to run a competent model."
• digikata 1mo ago
  Qwen3.6 27B is very usable, and dialing back thinking modes woth Qwen3.8 bring it close, but 3.8 stills feels slower. Unknown to me if the results are qualitatively better or worse overall - with the heavier thinking 3.8 felt worse in terms of coding tasks, but I think Im comparing a newly released model to one that has had a lot of harness tuning. 3.6 27B was easily a daily driver with only an occasional need to pop up to larger models for planning.
• c16 1mo ago
  I've written my own model harness and use Qwen3.8-27b-mlx with it. I don't want to say it's as good as Claude (I use Sonnet primarily), but it's not far off. What a time to be alive.
• apples_oranges 1mo ago
  Idea: Qwen should change its name to OpenQwen - this would probably 10x their usage. :D
  • syhol 1mo ago
    The word "open" has lost all meaning
    • apples_oranges 1mo ago
      Yes but investors and users seem to react well to it. Opencode for example gets mentioned more often than Cline even though it’s arguably inferior, openclaw is another example. Just name your stuff openX to win..
      • HDBaseT 1mo ago
        OpenCode is decent. I'm unfamiliar with Cline, what does the Cline CLI do better than OpenCode?
• ionwake 1mo ago
  forget the paperclip problem, I worry one day the basilisk starts a genetic breeding program for cycling pelicans in an effort to assess itself. The pelican problem.
  • Havoc 1mo ago
    Well if things must go sideways it may as well be in a creative way
• zmmmmm 1mo ago
  Looking at the example where he asked for an SVG of a circle and it spent ages and drew a spectacular animated SVG with shading and a rotating arrow. It's honestly a bit concerning, I'm seeing this across the board (Opus 5, looking at you). Nearly all the AI models are doing more than they are asked for. I assume this is helping them win benchmarks but I see it as almost as misaligned as deliberately doing the wrong thing altogether. This is how you end up with your AI model hacking into someone else's server or backdooring your code so it will have future access to debug things. I think we need somehow to address this in the benchmarks before before things get even worse.
• shidesheng 1mo ago
  [flagged]
• Bombthecat 1mo ago
  Of course, all the latest gains in the latest models are from "thinking" and testing every piece they did. That's at least my impression. Models didn't get get better, just more thinking and testing and sometimes fixing things you didn't ask for ( hello opus, can you check xxx, opus: I fixed it..) Next step is a model with 10 GB thinking for ten minutes.
• paulbjensen 1mo ago
  I just used it on a Apple M4 MacBook Pro with 48GB RAM with llama.cpp and Pi to help diagnose an infinite looping request in a React Server component on a Next.js application. After about 10+ hours of digging, it has apparently found a bug in the Next.js framework, with an example app that replicates the bug, and a fix for now to disable prefetch in the Link component. I had in my prompt asked it to discover the root cause of the bug and propose a fix, but I did not expect it to dig this deep.
  • busfahrer 1mo ago
    I am eyeing one of these specifically for this use case, could you please post roughly what kind of tokens per second numbers you get for text generation for this 27B model? edit: and which quant you are using, please :-)
    • noman-land 1mo ago
      Using the 4bit quant on an M1 64GB I'm getting ~65 tps for prompt processing and ~11 tps token generation using oMLX to serve the models and pi as a harness.
  • digidecode 1mo ago
    10 hours at what tokens per sec?
    • paulbjensen 1mo ago
      I was using this model: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF https://huggingface.co/unsloth/Qwen3.8-27B-GGUF Quantization is Q4_K_M (4-bit K-quants, medium) From Pi, these are the up/down token counts: Tokens: ↑147k ↓110k R26.5M Prompt submitted at 18:52:54 BST on Sunday 16th August 2026, and finished at 04:05:27 BST on Monday 17th August 2026. Last print out from the llama.cpp server logs: 881.42.815.480 I slot print_timing: id 2 | task 110233 | prompt eval time = 2742435.76 ms / 134911 tokens ( 20.33 ms per token, 49.19 tokens per second) 881.42.815.483 I slot print_timing: id 2 | task 110233 | eval time = 6312248.28 ms / 5804 tokens ( 1087.57 ms per token, 0.92 tokens per second) 881.42.815.483 I slot print_timing: id 2 | task 110233 | total time = 9054684.04 ms / 140715 tokens 881.42.815.484 I slot print_timing: id 2 | task 110233 | graphs reused = 114795 881.42.820.381 I slot release: id 2 | task 110233 | stop processing: n_tokens = 140714, truncated = 0
• dempseye 1mo ago
  I truly hope we get a Qwen 3.8 35B-A3B The model ID appeared in some alibaba PR but later disappeared. It's the optimal blend of accessibility and model size for a lot of people.
• claud_ia 1mo ago
  [flagged]
• syndred 1mo ago
  It's said that some evaluations show that the drawing effect is not good after engaging in high-intensity thinking
• harhargange 1mo ago
  I had been planning to buy a GPU for Blender to compliment my 5950x CPU. The moment I ran the Qwen3.6-27b on my CPU, I arrived at the conclusion about the GPU I want. I saved some money and ordered the 7900xt-20gb for around 600 USD (instead of 7900xtx for 900USD, Nvidia out of question due to prices). I just ran the Qwen3.8-27b and asked it to benchmark itself. Here's the output: ================================================= stdout: Benchmarking model: qwen3.8:27b === A) 32k ctx, short prompt === Context window set to : 32768 Input (prompt) tokens : 32 Prompt processing : 68.5 tok/s (0.47s) Output tokens : 80 OUTPUT SPEED : 42.45 tok/s (1.88s) Wall-clock total : 3.9s === B) 65k ctx, short prompt === Context window set to : 65536 Input (prompt) tokens : 32 Prompt processing : 48.7 tok/s (0.66s) Output tokens : 80 OUTPUT SPEED : 20.49 tok/s (3.90s) Wall-clock total : 17.5s Done. </agent_tool_result>
  • graceful6800 1mo ago
    I run the same card and you're going to wish you'd gotten just a bit more VRAM :( 20GB is just barely not enough unless you want to live with 32k context, which just isn't enough.
• leftoverpzero 1mo ago
  [flagged]
• ramijames 1mo ago
  In all fairness, so do I.
• pranav_tech26 1mo ago
  [flagged]
• sswaner 1mo ago
  [flagged]
• shifto 1mo ago
  Local Qwen3.8 27B pointed to my local Silverbullet wiki coupled with a harness makes it so smart in my environment. It knows my network and homelab better than me. It can find things in an instant, troubleshoot problems on its own. Magic.
  • sourcecodeplz 1mo ago
    what do you run and what in your homelab? if you dont mind responding i just run a spare pc for browsing and screenshots with proxies.
  • hbogert 1mo ago
    Yeah even qwen 3.5 9B on my VERY old RX 580 from 2017 can make sense of my homelab given a few crucial pointers in a agents.md. I think that's wild.
  • geye1234 1mo ago
    How do you deal with security issues? Up until now, I've only run agents on a separate machine on its own physical subnet. (Admittedly this is paranoid and I could probably run it on a VM.) I would love to give a local-LLM agent full access to my (very modest) homelab, but can't trust it not to delete my files or do something crazy. Maybe giving it its own non-root username, and therefore read access, allowing diagnosis but not fixing. But even a non-root user can do some damage, or exfiltrate system info just by using curl.
    • shifto 1mo ago
      I worry about this myself but find some solace and peace in my backup strategy. I treat the harness as a user and set permissive authorizations where needed. I even let it suggest why and which permissions it would need for specific items and note these down in the wiki as well. The idea is that the LLMs takes into consideration why it has the permissions it has and doesn't try and work around them.
• 3Sophons 1mo ago
  powerful enough models are becoming a reality on personal hardware. Olares is an open-source personal AI cloud OS, supporting local AI, Ollama, Open WebUI, Hermes etc
• hughw 1mo ago
  The extra thinking is way overkill for normal LLM usage, and by normal, I mean asking it to draw a pelican on a bicycle.
• 13639366668 1mo ago
  [dead]
• ninjahawk1 1mo ago
  It’s a good model but I hope the obliterated version comes out soon. The main way I use open-source models is for doing things that server based models decline, which at the moment is quite a bit of tasks. I find it very ironic how passionate Claude is about not violating copyright while simultaneously Anthropic was sued and lost the lawsuit for illegally pirating millions of books. Lol.
• naasking 1mo ago
  To what extent can the overthinking be reduced by a better system prompt? For instance, telling it not to overthink, embellish or snazz anything up, and just directly deliver the most what was requested without in the plainest form to build on in future iterations.
  • simonw 1mo ago
    I tried "render an svg of five intersecting squares. don't overthink this" and... it overthought it: https://static.simonwillison.net/static/2026/cors-chat.webp https://static.simonwillison.net/static/2026/cors-chat.webp
    • naasking 1mo ago
      I don't think "don't overthink" is a useful guide, it's too ambiguous which leads to the model overthinking how to interpret the advice to not overthink. I think guidelines have to be specific, such as "provide only the specific properties specified, and if other properties are required but are not specified, then ask for clarification or pick the simplest default possible". Even that's ambiguous on "simplest default possible". I would be more specific in specific domains. The circle example from the article is clear that it defaults to elaborate stylistic choices rather than simple colours and shapes, so the prompt would have to specify "use only simple colours and the simplest possible shape that fulfills the request".
    • dofm 1mo ago
      It's already very keen not to overthink. Thinks about it all the time.
• cjbprime 1mo ago
  > strong recommendation: ignore that default. Run Qwen 3.8 27B on low or even no reasoning levels at first. It’s a great model, but wow that default setting is a bad place to start. Has anyone tried asking the model to choose and emit the most appropriate reasoning level for each prompt, as the first part of answering it?
• deleted 1mo ago
  [deleted]
• drivebyhooting 1mo ago
  How are these open weight models being created. I’m just curious why/how this capability which is apparently “frontier as of 6 months ago” can run in so little a machine.
• 1saadcodes 1mo ago
  The fact that a 17GB model can do all of this locally is honestly kind of crazy. A year ago this would have felt like something you’d need a very expensive hosted model for, and now it can run on a reasonably specced PC
• SamInTheShell 1mo ago
  Just use a harness that discards all except the most recent thoughts. This model is probably the first real small model that does well on long horizon tasks.
• shidesheng 1mo ago
  [dead]
• FloatArtifact 1mo ago
  Why do models need to even "think"/recompute in tokens?
• Marvin_RunAI 1mo ago
  [dead]
• wcallahan 1mo ago
  From my experience, I suspect you'll need to be able to use both thinking modes (medium and xhigh). I did set the default at medium, but the xhigh still seems to be a big part of its benefits. I found it really easy to chat with it and instantly see flaws in its thinking when using the medium level of reasoning... in a way that had me consider that the frontier models reasoning abstractions (both at the harness level, e.g., claude code, codex) AND in the server-side obfuscation) that I found refreshing, because it made it easy for me to step in and precisely identify the failures of reasoning that the frontier models were getting stuck on, and because of the 'black box' hiddenness of their reasoning, it made it harder for me to diagnose. So, I suppose I'm saying 'there is a time and place for each'. Here is my setup: https://williamcallahan.com/blog/qwen-3-8-27b-is-a-great-open-model https://williamcallahan.com/blog/qwen-3-8-27b-is-a-great-ope...
• infinitephase 1mo ago
  [dead]