16 ms·
Qwen3.8-Flash-Next
https://imageat.com/models/qwen-3-8-27b-uncensored https://imageat.com/models/qwen-3-8-27b-uncensored
- whwhyb 2mo agolooks like it's better than deepseek v4 flash
- xvdvlinux 2mo agoI agreed
- freakynit 2mo agoThose benchmarks look seriously impressive.. considering how small of a MoE model this is.
- skarz 2mo agodo we really need breaking news about qwen posted every single day?
- iAMkenough 2mo agoyes there’s no shortage of online real estate
- tosh 2mo agothis is a new architecture (foreshadowing qwen 4) > trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board https://x.com/Alibaba_Qwen/status/2092591393424515114 https://x.com/Alibaba_Qwen/status/2092591393424515114
- pseudony 2mo agoI and presumably quite a few others with AMD AI or Apple Mac platforms are very impacted by this. :) It is very relevant and for a certain group of us, far more impactful to our work the next month(s) than any blog post could be.
- KronisLV 2mo agoIf there’s news, then yes. This is a pretty great new release for those still stuck on Qwen3.6 35B A3B if they have enough memory but don’t have super powerful compute. I wonder if I could get this running through vLLM on 6x Nvidia L4 - the 3.6 worked great on 4 cards but sadly TP6 just isn’t a thing and I don’t have 8 cards available, maybe it’s gonna be okay with like TP2 and MTP. I have no idea at this time, probably need to test out what even might be possible.
- NitpickLawyer 2mo agoThis particular release is interesting because it's a preview of qwen4 architecture. And, while benchmarks are iffy, this is a direct comparison, by the same team, with qwen3.8-27b that was pretty well received for a local model. This "next" release adds a new concept, first public release with n-grams, I think. And it's in a MoE size that is likely to be very fast and cheap to serve (faster than 27b for sure). It's also well suited for inference on alternative compute (i.e. sparks, macs, etc) so it's relevant to local users.
- dofm 2mo agoThis actually is meaningful news, I think. Pretty wide audience appeal in the local LLM space too.
- c16 2mo agoThere are many topics, personalities and politicians we hear about daily who have no merit. Qwen's advances do (currently) have merit.
- christkv 2mo agoLooks like a good model for strix halo
- tosh 2mo agothis is a new architecture (foreshadowing qwen 4) > trained at just 1/9 the cost of Qwen3.7-Plus, while outperforming it across the board https://x.com/Alibaba_Qwen/status/2092591393424515114 https://x.com/Alibaba_Qwen/status/2092591393424515114
- deleted 2mo ago[deleted]
- rohansood15 2mo agoDidn't expect it to beat 3.8 27B so cleanly. Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.
- Squarex 2mo agoI don't like these comparisons. Sure it is impressive, but it does not have a world knowledge of larger models. It has most of theirs intelligence.
- LaurensBER 2mo agoIf/when we can get larger context this will mostly be mitigated by these smaller models being able to search the internet. Self-learning/improving would be even better but that's still a long way to go.
- redox99 2mo agoSearch results suck because the web sucks these days. The big models from OpenAI/Anthropic have every book in existence baked into them
- jameshart 2mo agoI don’t think that’s the right way to think about LLM ‘knowledge’. They don’t have absolute recall of everything in the training set. They have been trained so that they have weights that can predict what those books might say - that is, if they read them they would find the contents unsurprising. That doesn’t mean it wouldn’t be helpful to pull relevant passages of text directly into context for a particular task.
- infp_arborist 2mo agoDoes it really matter? What about including all relevant and up-to-date literature as skills for local models? I have no experience with this but I am pretty sure someone has already thought about it.
- lnenad 2mo agoAdding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
- grim_io 2mo agoThat's low reasoning for a model, but max for a HN comment.
- cyanydeez 2mo agoMy stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat. Also, heating my home during the winter is nice. Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.
- lnenad 2mo agoYeah 122B is the sweet spot for me as well. Even deepseek flash overthinks on stuff way too much. I think they fully rely on large reasoning turns to achieve better quality. The result of course means we wait a long time to get results even with high throughput as a lot of tokens are wasted.
- redrix 2mo agoYou’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each: 1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment. One final thing worth mentioning: (I made myself irrationally angry writing this)
- gorgmah 2mo agoYou made me irrationally laugh reading this
- martinald 2mo agoFYI: nothing seems to be able to run this (easily) yet. llama.cpp, vllm etc I couldn't get working because of no support in the mainline version.
- kzrdude 2mo agoThey are giving pointers to how to run it now using for example https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next (and an especially provided vllm release).
- martinald 2mo agoah I don't think that page was up when I checked, it 404ed!
- a_humean 2mo agoProbably going to take a 1-3 days for support to land in llama.cpp and vllm.
- flounder3 2mo agoRelevant PR: https://github.com/ggml-org/llama.cpp/pull/27742 https://github.com/ggml-org/llama.cpp/pull/27742 This branch works now: https://github.com/unslothai/llama.cpp/tree/qwen4exp/qwen3.8-flash-next https://github.com/unslothai/llama.cpp/tree/qwen4exp/qwen3.8... cmake -B build -DGGML_CUDA=ON or cmake -B build -DGGML_METAL=ON then cmake --build build --config Release -j --target llama-server llama-cli
- amclennon 2mo agoIt looks like this also undercuts the already absurdly inexpensive Deepseek Flash in pricing. Wild.
- kzrdude 2mo ago(Edited: I thought Qwen3.8 Flash Next was smaller, but it's not, in bytes. Here's how they compare.) DSV4 Flash 304B params, 167 GB download (at full size) Qwen3.8 Flash Next 180B params, 360 GB download (at full size)
- geooff_ 2mo agoWhere are you seeing that? At the bottom of this post from Qwen I see: Qwen 3.8 flash: $0.16 / $0.47 Compared to Deepseek 0723: $0.03 / $0.075 (units in USD/m tok)
- twohaibei 2mo ago0.03 / 0.075 ? Where can i get that prices? Especially during peak hours DS4flash became much more money hungry than last month. https://api-docs.deepseek.com/quick_start/pricing https://api-docs.deepseek.com/quick_start/pricing
- kaelwd 2mo agohttps://openrouter.ai/deepseek/deepseek-v4-flash-0731?endpoint=f954a48e-1c90-433b-9348-4720f8030331#pricing https://openrouter.ai/deepseek/deepseek-v4-flash-0731?endpoi... 8t/s though apparently and their cache hit rate is terrible so I don't think it's worth it over Relace.
- pram 2mo agoIt's in Unsloth Desktop already. Looks like it's 73GB, so 128GB Mac or Strix Halo etc will work. Exciting!
- cwizou 2mo agoDownload is available, but likely need to wait for an update, I get this which is understandable with the architectural change : Original error: llama.cpp does not support this GGUF's model architecture ('qwen4exp') Edit : Saw the pull request, should arrive soon enough https://github.com/ggml-org/llama.cpp/pull/27742 https://github.com/ggml-org/llama.cpp/pull/27742
- andy99 2mo agoI only see a 1-bit quant posted on unsloth HF and it’s 72.5 GB. Is that what you mean? That’s much bigger than I expected. If you can’t run a 4 bit quant in on Strix Halo it becomes a lot less interesting. https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
- agile-gift0262 2mo agoIn their page they say it will need at least 112GB[0], so including context, that would be a tight fit. I'm also hoping I can make a q4 fit on my 128GB strix halo [0]: https://unsloth.ai/docs/models/qwen3.8-next#qwen3.8-flash-next-requirements https://unsloth.ai/docs/models/qwen3.8-next#qwen3.8-flash-ne...
- walrus01 2mo agoin llama-server PR 27742 it fits fine in 128GB RAM on a CPU only system , this is with --load-mode mlock to stuff the whole thing persistently into memory at llama-server launch time, no mmap 0.01.033.250 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted | 0.01.033.261 I common_memory_breakdown_print: | - Host | 118186 = 106166 + 8898 + 3122 | 0.01.092.684 I common_params_fit_impl: projected to use 118186 MiB of host memory vs. 128855 MiB of total host memory
- KolmogorovComp 2mo agoWill this be cheaper than DS4flash ?
- armcat 2mo agoHow is input token efficiency/verbosity on this model? Has anyone tried? GLM 5.2 was doing lot of turns and thinking piling up input tokens in the context (compared to Claude and GPT models). Then Qwen3.8-27B was 2x of that. Both delivered good output results but those cumulative input token costs were not cheap. Note this is on our specific business workloads. Genuinely interested in other people's experience (if you are able to try it out).
- petu 2mo agoHaven't tried, would be surprised if it's any different. It's new arch demo for future Qwen 4 family, but (as I understand) training recipe/data is same as any other 3.8 model.
- a_humean 2mo agoWaiting for llama.cpp support to land, but this might be a big deal for Strix Halo users. 6B active params helps around the memory bandwidth constraints, but a 128GB box can probably run the Q3/Q4 quants fairly easily with a decent context size. This might actually be better for strix users than 27B, which was already very good.
- hedgehog 2mo agoIn my early testing it's way better both quality and speed on Strix Halo (posted recipe in sibling comment).
- GrinningFool 2mo agoUsing llama.cpp I one-shotted (2 hours) a reasonable asteroids clone on my strix halo/128 using the 1 bit quant, using my custom harness (which isn't anything exceptional). It was tedious - a lot of second guessing itself, and quadruple checking things it fixed a couple of iterations back - but it got there and the result is a playable game. Speed starts out strong, but definitely drops off as context grows. At the end (I think context about 70k) it was down to 12 output tps. Mind a bit blown.
- Imustaskforhelp 2mo agoPelican: https://gist.github.com/SerJaimeLannister/8fdef9c00175da0ca6cd97f158b8f14d?permalink_comment_id=6334222#gistcomment-6334222 https://gist.github.com/SerJaimeLannister/8fdef9c00175da0ca6... Aside from the pelican, I am sort of impressed by the fact that things are going the way in terms of really impressive small models. Also I love how this uses N-gram embedding. I think that Longcat was the first one who used it (I submitted that submission on hackernews because I really just loved the idea of it that I understood), I am certainly more interested in local LLM models and its interesting how they are utilizing new architectures to do some really impressive optimizations! (Do note that I created it using a free rate limited end-point that I found on the huggingface space section: https://victor-chat-with-qwen3-8-flash-next.hf.space https://victor-chat-with-qwen3-8-flash-next.hf.space)
- stymaar 2mo ago> I think that Longcat was the first one who used it Wasn't it introduced by Gemma?
- andai 2mo agoFather, I cannot scroll the website.
- loclol101 2mo agoDefinitely need to try this out locally.
- myshapeprotocol 2mo ago[dead]
- NooneAtAll3 2mo agowhat's the deal with absent scrollbar on the website?
- andy99 2mo ago> Qwen3.8-Flash-Next features a 125B-parameter main model, supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. Didn’t see this mentioned yet. I wonder what this means for the effective size. It’s evidently ~176B paramètres, but how does that get quantized. A 4-bit quant under 100GB seems unlikely, I’m suspecting this won’t run in 128GB unified memory In principle I like the idea of trading more memory for compute though, even if there’s a memory shortage right now
- khalic 2mo agoGonna have to wait a few days to see what the wizards of the HF community come up with…
- Phemist 2mo agoThey are already working on it. https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF https://unsloth.ai/docs/models/qwen3.8-next https://unsloth.ai/docs/models/qwen3.8-next > You will need at least 75 GB of RAM or unified memory to run the model. Its smallest 1-bit quantized version is larger than usual because of the model’s architecture so 1-bit isn't really 1-bit at all. However, this also means the quantization is less aggressive, allowing the model to retain more of its original accuracy than more heavily quantized models. Lots of RAM required even for the 1-bit, which is already downloadable. Interested to see how well this one works compared to Ornith1.5-35B-A3B I've been running (and quite happy about). Edit: but llama-cpp does not yet support it.
- garo-pro 2mo agoInterestingly they also share the parameter count for Qwen 3.7 Plus (397 b a17b). I don't think these were known before but I might be wrong.
- kzrdude 2mo agoI guess it was obscure before, but someone did "know" it: https://old.reddit.com/r/Qwen_AI/comments/1u7xnvq/how_big_is_qwen_37_plus_do_we_have_any_idea_37_max/osd1lux/ https://old.reddit.com/r/Qwen_AI/comments/1u7xnvq/how_big_is...
- anon373839 2mo agoDoes anyone have an idea how this might perform on a DGX Spark at longer contexts? I've been trying to investigate their performance with these medium-sized MoE models, but I'm seeing a lot of incomplete and conflicting information. The 273 GB/s bandwidth looks awfully bad on paper...
- rohansood15 2mo agoGiven the new architecture, speeds are harder to estimate. Max would be ~40 tok/s.
- lxkarthi 2mo agoI can run Deepseek flash 0731 version (ds4, esl3) on single DGX spark. getting around ~20 tok/s. It's great. Quantized version of this model would probably run on the DGX spark. I am excited to wait for quantized models that fits in single DGX spark.
- colingauvin 2mo agoA single Spark alone is not worth the price. You are paying $1000 just for networking equipment you aren't using. At 2x it starts to maybe become worth it if you don't want to deal with Apple. Outside of the newest Macs, I can't think of anything else you can get 256 GB ~550 GB/s memory bandwith for $8200. Even at 3-4 Sparks it scales relatively well. With 2x Sparks, I am getting 40 t/s. I'd guess that without MTP you'd get 12-15 on 1 Spark, maybe 20 with MTP?
- railka 2mo agoAlso announced GLM-5.3-Flash: https://news.ycombinator.com/item?id=49449507 https://news.ycombinator.com/item?id=49449507
- axegon_ 2mo agoAaaaaaaaaaaaaand dario meltdown on twitter in 3, 2, 1...
- khangtong988 2mo agoYoh yoh I like It
- lucabytheway 2mo agovery interesting. new architectures is the most interesting type of news. after what i experienced when gpt-oss came out i have been on the look out for architectural approaches that improves efficiency.
- Roark66 2mo agoUnsloth doesn't have all quant versions yet :-(
- schopra909 2mo agoCan someone explain the intuition behind the en-gram idea? I know DeepSeek published a paper about it a few months ago and the Gemma models have a lightweight version of it; but it hasn’t clicked for me yet
- deleted 2mo ago[deleted]
- a11r 2mo agoQuoting RGFusion from Reddit: LLMs run into an issue where the further you train a model, the more it overwrites facts with generalized concepts. You need the model to be able to do both. Intelligence arises from generalization, but without accurate information the model will hallucinate. The engram table allows for a low-computational method of fact-recall. You can think of it like a better form of RAG, where the data doesn't take up any of your context window and it's injected deeper into the model's layers, freeing the lower layers to carry out abstraction. This results in better "focus" for the model, both in regards to its intelligence and context recall. Basically, they've separated the specificity-critical portions of the models memory into a parameter space that doesn't need fast compute (you can run it on system RAM) and allows the model to be trained on higher volumes of data without ruining its knowledge-base. https://www.reddit.com/r/LocalLLaMA/comments/1vy6smx/comment/p5uge72/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button https://www.reddit.com/r/LocalLLaMA/comments/1vy6smx/comment...
- TonyStr 2mo agoVery interesting. Is this compatible with MoE architectures? I'm not too familiar with how this works.
- stefan_ 2mo agoI think these "Flash" models are sort of an evolutionary dead end. Sure, there are some routine tasks and applications where they can be used. But for the actual novel development work? It's much better to run a big model at high power for 30 mins than watch the Flash model struggle for 2 hours and produce massive churn. Same reason your phone has a few big CPU cores for real work, it's much better to "race to idle" than have an "efficient" core struggle. Shitty experience, shitty power efficiency.
- wolttam 2mo agoIt depends how you use the models. These small models work great for developers who prefer to stay more in the loop, and only task the model with things that can really only be interpreted in one way. Not to mention, they’re great for self-hosting and getting yourself to not be dependent on some API that can go down or be altered at any time. Big models seem to mostly be good for pushing ahead the frontier - the smaller models tend to gain the frontier’s capabilities after only a handful of months anyway. Many are perfectly content remaining a few months behind the bleeding edge.
- jononor 2mo agoIf you have good feedback signals, like tests/benchmarks/etc, then it is potentially better to do multiple turns where model uses that to adjust code. Which might not need as smart a model.
- deleted 2mo ago[deleted]
- simonw 2mo agoI ran some pelicans at the four different reasoning levels (none, low, medium, xhigh - apparently high and xhigh are aliases of each other) on a DGX Spark using Unsloth's unsloth/Qwen3.8-Flash-Next-GGUF (UD-IQ1_S): https://tools.simonwillison.net/markdown-svg-renderer#url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Ff9c69ebdab90d8a45b8de4742cc7b840 https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Surprised I didn't get one I liked as much as the Qwen 3.8 27B one https://simonwillison.net/2026/Aug/16/qwen-38-27b/#the-default-of-extra-high-results-in-spectacular-over-thinking https://simonwillison.net/2026/Aug/16/qwen-38-27b/#the-defau... , maybe because of quantization.
- kzrdude 2mo agoIf I read correctly, that's based on a 1-bit quantization, and can we really expect that to produce any useful output at all?
- walrus01 2mo agoif the goal is vector art salvador dali, like melting clocks and stuff, sure
- coder543 2mo agoThe spark can easily run UD-Q4_K_XL on this model... using IQ1_S doesn't make much sense.
- badatnames 2mo agoYou should find an excuse to offer 3D printed extruded pelicans from various models as awards for something. I have no idea for what, but the idea captivates and I'd love to win one somehow. They'd be collector's items in a few decades
- jononor 2mo agoIf Simon would pitch for example PCBWay that and I am pretty sure they will sponsor it (assuming their logo stays). They can do laser engraved versions also ;)
- respectattentio 2mo agoI can't imagine the future any more. US companies playing it safe and control models releases. Chinese companies are just like open source everything. It's like Chinese are incentivized to open source from day one (years ago). While most US companies are deciding in realtime. It's crazy that we need both to survive and advance further in the future we have never imagined.
- cindyllm 2mo ago[dead]
- hedgehog 2mo agoIn initial testing on Ryzen 395 / Strix Halo it's about 22 tokens/s generation and the output quality is impressive. Better and faster than 3.8 27B, and enough better to justify moving away from 3.6 35B even though 35B is still faster. The Unsloth weights don't come with vision support but you can add that back yourself. llama.cpp recipe in case anyone else wants to save some time on setup: https://pastebin.com/fcqsbDTv https://pastebin.com/fcqsbDTv
- mdp2021 2mo ago> Better... than 3.8 27B How can a "6b active per token" MoE be better than a just released 27b in the same family? Possible maybe, but quite interesting and requiring some explanation. Edit: ok, on second thought, probably because the small "experts" are really rich in specialized training compared to the dense model. Still raising questions about the details, e.g. the reasoning abilities (or all meta-skills) of a "6b per token" network compared to the dense capabilities...
- hedgehog 2mo agoSeparating knowledge from reasoning so you only pay for what you use is a big rationale for MoE, the big problem being MoE training has historically been hard to get right. In a dense model every single token you're paying a cost to determine whether you're now talking about the flavor of durian.
- mdp2021 2mo agoSure, the "6b subset" can be more knowledgeable on its area than a whole 27b generalist (and more efficient), but where is the simulated Intelligence encoded? A 6b subset as or more intelligent than a 27b raises the question of how metacognition skills are stored.
- mapontosevenths 2mo agoMOEs are built by training a second "router" model to identify which parts matter inside the dense model. Think of MOE as ignoring noise, rather than a more efficient encoding of the data we teach it which ones can be ignored. Turning down the noise actually sharpens the results sometimes. Modern LLM's are wildly inefficient.
- xlayn 2mo agoFor the impatient, I merged llama.cpp tentative branches to get it running here https://github.com/alainnothere/llama.cpp/tree/disk-cache-eviction https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev..., thing runs at 23.54 token/sec and my setup runs at high 30 the 3.8 dense 27B. and this is the pelican from the iq4_xs model https://github.com/alainnothere/llama.cpp/blob/disk-cache-eviction/media/pelicanRidingBycicle.jpg https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...
- cmrdporcupine 2mo agoIf you've got a DGX Spark try my little engine: https://github.com/rdaum/eider/ https://github.com/rdaum/eider/ NVFP4 quant
- cmrdporcupine 2mo agoIf you have a DGX Spark, try my Spark/SM12x specific inference engine. I've got it (Qwen 3.8 flash next) working (sans ... MTP working on that now). https://github.com/rdaum/eider/ https://github.com/rdaum/eider/ ~80tok/sec prefill, 12tok/sec decode, ~80GiB memory resident, the n-gram table pages from SSD.
- jedisct1 2mo agoMLX quants for Apple M5 with 128 GiB RAM: https://huggingface.co/jedisct1/Qwen3.8-Flash-Next-oQ4e-128k https://huggingface.co/jedisct1/Qwen3.8-Flash-Next-oQ4e-128k
- tristor 2mo agoLooks like it errors out in LM Studio using the Unsloth quants, apparently the Unsloth team has already posted patches for llama.cpp to support this.
- deleted 2mo ago[deleted]
- potus_kushner 2mo agoat first this looked like something one could run on CPU with 64GB RAM with a 2-3 bit quant, at possibly half the speed of 3.6 35B-A3B, however the 50B ngram sidecar makes it impossible. and oddly, unsloth's page lists the ngrams as 50GB even though they say it's in 4 bits. should be 25GB according to my math. anyway, the new ngram architecture makes it pretty much unusable for regular folks who cant afford more than 32-64 GB ram in this RAMocalypse.
- hadlock 2mo agoI suspect we will see optimizations where the various vectors of the n-gram you actually use are hot in vram, the rest are warm in system memory and then cold storage on nvme. Same with MoE. If your workflow is particularly same-y then you're looking at cache miss below 5% with NTP/MTP turned on and the right harness. Agentic "openclaw" type stuff cache miss might be below 1% in the right local llm setups. There's been zero exploitation of n-gram stuff yet, it will be very interesting as things progress.
- postal6666 2mo ago[dead]
- monster_truck 2mo agoI'm really impressed. Gave QwenCloud $18, handed 3.8-flash a few big forks of a lot of code, it did some archeology and made a clean merge. Then it used the project's tools to bisect a regression and fix it. Was not expecting it to just get that right without any fuss, and it barely used 10% of this weekly limit. Something like 90M cached in/400k out for $0.45 is wild
- seobotaicom 2mo ago[dead]
- hasanbasbunar 2mo ago[flagged]