12 ms·
Qwen3.5: Towards Native Multimodal Agents
- danielhanchen 8mo agoFor those interested, made some MXFP4 GGUFs at https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF and a guide to run them: https://unsloth.ai/docs/models/qwen3.5 https://unsloth.ai/docs/models/qwen3.5
- plagiarist 8mo agoAre smaller 2/3-bit quantizations worth running vs. a more modest model at 8- or 16-bit? I don't currently have the vRAM to match my interest in this
- jncraton 8mo ago2 and 3 bit is where quality typically starts to really drop off. MXFP4 or another 4-bit quantization is often the sweet spot.
- deleted 8mo ago[deleted]
- AbstractGeo 8mo agoIMO, they're worth trying - they don't become completely braindead at Q2 or Q3, if it's a large enough model, apparently. (I've had surprisingly decent experience with Q2 quants of large-enough models. Is it as good as a Q4? No. But, hey - if you've got the bandwidth, download one and try it!) Also, don't forget that Mixture of Experts (MoE) models perform better than you'd expect, because only a small part of the model is actually "active" - so e.g. a Qwen3-whatever-80B-A3B would be 80 billion total, but 3 billion active- worth trying if you've got enough system ram for the 80 billion, and enoguh vram for the 3.
- zozbot234 8mo agoYou don't even need system RAM for the inactive experts, they can simply reside on disk and be accessed via mmap. The main remaining constraints these days will be any dense layers, plus the context size due to KV cache. The KV cache has very sparse writes so it can be offloaded to swap.
- nl 8mo agoAre there any benchmarks (or even vibes!) about the token/second one can expect with this strategy?
- zozbot234 8mo agoNo real fixed benchmarks AIUI since performance will then depend on how much extra RAM you have (which in turn depends on what queries you're making, how much context you're using etc.) and how high-performance your storage is. Given enough RAM, you aren't really losing any performance because the OS is caching everything for you. (But then even placing inactive experts in system RAM is controversial: you're leaving perf on the table compared to having them all in VRAM!)
- cpburns2009 8mo agoIn my short testing on a different MoE model, it does not perform well. I tried running Kimi-K2-Thinking-GGUF with the smallest unsloth quantization (UD-TQ1_0, 247 GB), and it ran at 0.1 tps. According to its guide, you should expect 5 tps if the whole model can fit into RAM+VRAM, but if mmap has to be used, then expect less than 1 tps which matches my test. This was on a Ryzen AI Max+ 395 using ~100 GB VRAM.
- zardo 8mo agoThat's NVME storage in your test?
- cpburns2009 8mo ago
- doctorpangloss 8mo agoSimply and utterly impossible to tell in any objective way without your own calibration data, in which case, make your own post trained quantized checkpoints anyway. That said, millions of people out there make technical decisions on vibes all the time, and has anything bad happened to them? I suppose if it feels good to run smaller quantizations, do it haha.
- ggcr 8mo agoFrom the HuggingFace model card [1] they state: > "In particular, Qwen3.5-Plus is the hosted version corresponding to Qwen3.5-397B-A17B with more production features, e.g., 1M context length by default, official built-in tools, and adaptive tool use." Anyone knows more about this? The OSS version seems to have has 262144 context len, I guess for the 1M they'll ask u to use yarn? [1] https://huggingface.co/Qwen/Qwen3.5-397B-A17B https://huggingface.co/Qwen/Qwen3.5-397B-A17B
- danielhanchen 8mo agoUnsure but yes most likely they use YaRN, and maybe trained a bit more on long context maybe (or not)
- NitpickLawyer 8mo agoYes, it's described in this section - https://huggingface.co/Qwen/Qwen3.5-397B-A17B#processing-ultra-long-texts https://huggingface.co/Qwen/Qwen3.5-397B-A17B#processing-ult... Yarn, but with some caveats: current implementations might reduce performance on short ctx, only use yarn for long tasks. Interesting that they're serving both on openrouter, and the -plus is a bit cheaper for <256k ctx. So they must have more inference goodies packed in there (proprietary). We'll see where the 3rd party inference providers will settle wrt cost.
- ggcr 8mo agoThanks, I've totally missed that It's basically the same as with the Qwen2.5 and 3 series but this time with 1M context and 200k native, yay :)
- mynti 8mo agoDoes anyone know what kind of RL environments they are talking about? They mention they used 15k environments. I can think of a couple hundred maybe that make sense to me, but what is filling that large number?
- yorwba 8mo agoEvery interactive system is a potential RL environment. Every CLI, every TUI, every GUI, every API. If you can programmatically take actions to get a result, and the actions are cheap, and the quality of the result can be measured automatically, you can set up an RL training loop and see whether the results get better over time.
- radarsat1 8mo ago> and the quality of the result can be measured automatically this part is nontrivial though
- robkop 8mo agoRumours say you do something like: Download every github repo -> Classify if it could be used as an env, and what types -> Issues and PRs are great for coding rl envs -> If the software has a UI, awesome, UI env -> If the software is a game, awesome, game env -> If the software has xyz, awesome, ... -> Do more detailed run checks, -> Can it build -> Is it complex and/or distinct enough -> Can you verify if it reached some generated goal -> Can generated goals even be achieved -> Maybe some human review - maybe not -> Generate goals -> For a coding env you can imagine you may have a LLM introduce a new bug and can see that test cases now fail. Goal for model is now to fix it ... Do the rest of the normal RL env stuff
- NitpickLawyer 8mo agoThe real real fun begins when you consider that with every new generation of models + harnesses they become better at this. Where better can mean better at sorting good / bad repos, better at coming up with good scenarios, better at following instructions, better at navigating the repos, better at solving the actual bugs, better at proposing bugs, etc. So then the next next version is even better, because it got more data / better data. And it becomes better... This is mainly why we're seeing so many improvements, so fast (month to month, from every 3 months ~6 monts ago, from every 6 months ~1 year ago). It becomes a literal "throw money at the problem" type of improvement. For anything that's "verifiable" this is going to continue. For anything that is not, things can also improve with concepts like "llm as a judge" and "council of llms". Slower, but it can still improve.
- isusmelj 8mo agoIs it just me or is the page barely readable? Lots of text is light grey on white background. I might have "dark" mode on on Chrome + MacOS.
- dryarzeg 8mo agoI'm using Firefox on Linux, and I see the white text on dark background. > I might have "dark" mode on on Chrome + MacOS. Probably that's the reason.
- Jacques2Marais 8mo agoYes, I also see that (also using dark mode on Chrome without Dark Reader extension). I sometimes use the Dark Reader Chrome extension, which usually breaks sites' colours, but this time it actually fixes the site.
- thunfischbrot 8mo agoThat seems fine to me. I am more annoyed at the 2.3MB sized PNGs with tabular data. And if you open them at 100% zoom they are extremely blurry. Whatever workflow lead to that?
- nsb1 8mo agoWho doesn't like grey-on-slightly-darker-grey for readability?
- dcre 8mo agoYeah, I see this in dark mode but not in light mode.
- deleted 8mo ago[deleted]
- bertili 8mo agoLast Chinese new year we would not have predicted a Sonnet 4.5 level model that runs local and fast on a 2026 M5 Max MacBook Pro, but it's now a real possibility.
- lostmsu 8mo agoWill 2026 M5 MacBook come with 390+GB of RAM?
- bertili 8mo agoMost certainly not, but the Unsloth MLX fits 256GB.
- embedding-shape 8mo agoCurious what the prefilled and token generation speed is. Apple hardware already seem embarrassingly slow for the prefill step, and OK with the token generation, but that's with way smaller models (1/4 size), so at this size? Might fit, but guessing it might be all but usable sadly.
- regularfry 8mo agoThey're claiming 20+tps inference on a macbook with the unsloth quant.
- embedding-shape 8mo agoYeah, I'm guessing the Mac users still aren't very fond of sharing the time the prefill takes, still. They usually only share the tok/s output, never the input.
- alex43578 8mo agoQuants will push it below 256GB without completely lobotomizing it.
- Alifatisk 8mo agoWow, the Qwen team is pushing out content (models + research + blogpost) at an incredible rate! Looks like omni-modals is their focus? The benchmark look intriguing but I can’t stop thinking of the hn comments about Qwen being known for benchmaxing.
- trebligdivad 8mo agoAnyone else getting an automatically downloaded PDF 'ai report' when clicking on this link? It's damn annoying!
- ddtaylor 8mo agoDoes anyone know the SWE bench scores?
- jug 8mo agoIt's in the post?
- ddtaylor 8mo agoSorry, what I meant is if third party has them in their leaderboards. I don't usually trust most of what any of these vendors claim in their release notes without a third party. I know it says "verified" there, but I don't see were the SWE bench results are from a third party, whereas for the "HLE-Verified" they do have a citation to Hugging Face. I was looking for something closer to: https://www.vals.ai/benchmarks/swebench https://www.vals.ai/benchmarks/swebench
- mkl 8mo ago"SWE-Bench Verified" is the name of the benchmark: https://dev.to/duplys/swe-bench-swe-bench-verified-benchmarks-1cm https://dev.to/duplys/swe-bench-swe-bench-verified-benchmark.... Same with "HLE-Verified". It's nothing to do with third party testing. The citation you point to makes that clear.
- simonw 8mo agoPelican is OK, not a good bicycle: https://gist.github.com/simonw/67c754bbc0bc609a6caedee16fef89e8?permalink_comment_id=5989367#gistcomment-5989367 https://gist.github.com/simonw/67c754bbc0bc609a6caedee16fef8...
- tarruda 8mo agoAt this point I wouldn't be surprised if your pelican example has leaked into most training datasets. I suggest to start using a new SVG challenge, hopefully one that makes even Gemini 3 Deep Think fail ;D
- jon-wood 8mo agoI think we’re now at the point where saying the pelican example is in the training dataset is part of the training dataset for all automated comment LLMs.
- Mossly 8mo agoIt's quite amusing to ask LLMs what the pelican example is and watch them hallucinate a plausible sounding answer. --- Qwen 3.5: "A user asks an LLM a question about a fictional or obscure fact involving a pelican, often phrased confidently to test if the model will invent an answer rather than admitting ignorance." <- How meta Opus 4.6: "Will a pelican fit inside a Honda Civic?" GPT 5.2: "Write a limerick (or haiku) about a pelican." Gemini 3 Pro: "A man and a pelican are flying in a plane. The plane crashes. Who survives?" Minimax M2.5: "A pelican is 11 inches tall and has a wingspan of 6 feet. What is the area of the pelican in square inches?" GLM 5: "A pelican has four legs. How many legs does a pelican have?" Kimi K2.5: "A photograph of a pelican standing on the..." --- I agree with Qwen, this seems like a very cool benchmark for hallucinations.
- ertgbnm 8mo agoI'm guessing it has the opposite problem of typical benchmarks since there is no ground truth pelican bike svg to over fit on. Instead the model just has a corpus of shitty pelicans on bikes made by other LLMs that it is mimicking. So we might have an outer alignment failure.
- gunalx 8mo agoSad to not see smaller distills of this model being released alongside the flaggship. That has historically been why i liked qwen releases. (Lots of diffrent sizes to pick from from day one)
- woadwarrior01 8mo agoJudging by the code in the HF transformers repo[1], smaller dense versions of this model will most likely be released at some point. Hopefully, soon. [1]: https://github.com/huggingface/transformers/tree/main/src/transformers/models/qwen3_5 https://github.com/huggingface/transformers/tree/main/src/tr...
- exe34 8mo agoI get the impression the multimodal stuff might make it a bit harder?
- kpw94 8mo agoPer https://github.com/QwenLM/Qwen3.5 https://github.com/QwenLM/Qwen3.5, more are coming: > News > 2026-02-16: More sizes are coming & Happy Chinese New Year!
- deleted 8mo ago[deleted]
- dash2 8mo agoYou'll be pleased to know that it chooses "drive the car to the wash" on today's latest embarrassing LLM question.
- WithinReason 8mo agoIs that the new pelican test?
- dainiusse 8mo agoNo, this is "AGI test" :D
- giancarlostoro 8mo agoHave we even agreed on what AGI means? I see people throw it around, and it feels like AGI is "next level AI that isn't here yet" at this point, or just a buzzword Sam Altman loves to throw around.
- manmal 8mo agoI guess AGI is reached, then. The SOTA models make fun of the question.
- BlackLotus89 8mo agoIt's > "I want to wash my car. The car wash is 50m away. Should I drive or walk?" And some LLMs seem to tell you to walk to the carwash to clean your car... So it's the new strawberry test Edit https://news.ycombinator.com/item?id=47031580 https://news.ycombinator.com/item?id=47031580
- rfoo 8mo ago[flagged]
- zozbot234 8mo agoMy OpenClaw AI agent answered: "Here I am, brain the size of a planet (quite literally, my AI inference loop is running over multiple geographically distributed datacenters these days) and my human is asking me a silly trick question. Call that job satisfaction? Cuz I don't!"
- tarruda 8mo agoWould love to see a Qwen 3.5 release in the range of 80-110B which would be perfect for 128GB devices. While Qwen3-Next is 80b, it unfortunately doesn't have a vision encoder.
- bytesandbits 8mo agomaybe a deepseek v4 distill. give it a few days
- PlatoIsADisease 8mo agoWhy 128GB? At 80B, you could do 2 A6000s. What device is 128gb?
- vladovskiy 8mo agoGuess, it is mac m series
- the_pwner224 8mo agoAMD Strix Halo / Ryzen AI Max+ (in the Asus Flow Z13 13 inch "gaming" tablet as well as the Framework Desktop) has 128 GB of shared APU memory.
- hedgehog 8mo agoKeep in mind most of the Strix Halo machines are limited to 10Gbe networking at best.
- paulsmal 8mo agoyou can use separate network adapter with RoCEv2/RDMA support like Intel E810
- hedgehog 8mo agoMost Ryzen 395 machines don't have a PCI-e slot for that so you're looking at an extension from an m.2 slot or Thunderbolt (not sure how well that will work, possibly ok at 10Gb). Minisforum has a couple newly announced products, and I think the Framework desktop's motherboard can do it if you put it in a different case, that's about it. Hopefully the next generation has Gen5 PCIe and a few more lanes.
- lollobomb 8mo ago[flagged]
- Zetaphor 8mo agoWhy is this important to anyone actually trying to build things with these models
- soulofmischief 8mo agoIt's a rhetorical attempt to point out that we cannot trade a little convenience for getting locked into a future hellscape where LLMs are the typical knowledge oracle for most people, and shape the way society thinks and evolves due to inherent human biases and intentional masking trained into the models. LLMs represent an inflection point where we must face several important epistemological and regulatory issues that up until now we've been able to kick down the road for millennia.
- ghywertelling 8mo agoInformation is being erased from Google right now. Things which were searching few years ago are totally not findable at all now. One who controls the present can control both the future and the past.
- Zetaphor 8mo agoDid you know that you can do additional fine-tuning on this model to further shape its biases? You can't do that with proprietary models, you take what Anthropic or OpenAI give you and be happy. I'm so tired of seeing this exact same response under EVERY SINGLE release from a Chinese lab. At this point it's starting to read more xenophobic and nationalist than having anything to do with the quality of the model or its potential applications. If you're just here to say the exact same thoughtless line that ends up in triplicate under every post then please at least have an original thought and add something new to the conversation. At this point it's just pointless noise and it's exhausting.
- 8mo ago
- Matl 8mo agoIs it just me or are the 'open source' models increasingly impractical to run on anything other than massive cloud infra at which point you may as well go with the frontier models from Google, Anthropic, OpenAI etc.?
- regularfry 8mo agoIf "local" includes 256GB Macs, we're still local at useful token rates with a non-braindead quant. I'd expect there to be a smaller version along at some point.
- doodlesdev 8mo agoYou still have the advantage of choosing on which infrastructure to run it. Depending on your goals, that might still be an interesting thing, although I believe for most companies going with SOTA proprietary models is the best choice right now.
- segmondy 8mo agodepends on what you mean by impractical. but some of us are trodding quite along.
- mudkipdev 8mo agoIt's because their target audience are enterprise customers who want to use their cloud hosted models, not local AI enthusiasts. Making the model larger is an easy way to scale intelligence.
- vessenes 8mo agoGreat benchmarks, qwen is a highly capable open model, especially their visual series, so this is great. Interesting rabbit hole for me - its AI report mentions Fennec (Sonnet 5) releasing Feb 4 -- I was like "No, I don't think so", then I did a lot of googling and learned that this is a common misperception amongst AI-driven news tools. Looks like there was a leak, rumors, a planned(?) launch date, and .. it all adds up to a confident launch summary. What's interesting about this is I'd missed all the rumors, so we had a sort of useful hallucination. Notable.
- jorl17 8mo agoYeah, I opened their page, got an instantly downloaded PDF file (creepy!) and it's talking about Sonnet 5 — wtf!? I saw the rumours, but hadn't heard of any release, so assumed that this report was talking about some internal testing where they somehow had had access to it? Bizarre
- ranguna 8mo agoAlready on open router, prices seem quite nice. https://openrouter.ai/qwen/qwen3.5-plus-02-15 https://openrouter.ai/qwen/qwen3.5-plus-02-15
- esafak 8mo agono caching yet
- XCSme 8mo agoLet's see what Grok 4.20 looks like, not open-weight, but so far one of the high-end models at real good rates.
- sasidhar92 8mo agoGoing by the pace, I am more bullish that the capabilities of opus 4.6 or latest gpt will be available under 24GB Mac
- Someone1234 8mo agoCurrent Opus 4.6 would be a huge achievement that would keep me satisfied for a very long time. However, I'm not quite as optimistic from what I've seen. The Quants that can run on a 24 GB Macbook are pretty "dumb." They're like anti-Thinking models; making very obvious mistakes and confusing themselves. One big factor for local LLMs is that large context windows will seemingly always require large memory footprints. Without a large context window, you'll never get that Opus 4.6-like feel.
- azinman2 8mo agoDoes anyone else have trouble loading from the qwen blogs? I always get their placeholders for loading and nothing ever comes in. I don’t know if this is ad blocker related or what… (I’ve even disabled it but it still won’t load)
- HnUser12 8mo agoI’m on Safari iOS. I had to do “reduce other privacy protections” to get it to load.
- collinwilkins 8mo agoat this point it seems every new model scores within a few points of each other on SWE-bench. the actual differentiator is how well it handles multi-step tool use without losing the plot halfway through and how well it works with an existing stack
- XCSme 8mo agoI just started creating my own benchmarks (very simple questions for humans but tricky for AI, like how many r's in strawberry kind of questions, still WIP). Qwen3.5 is doing ok on my limited tests: https://aibenchy.com https://aibenchy.com
- Western0 8mo agoWho can tell me how creating a sound generate from text localy
- cpburns2009 8mo agoYou're looking for text-to-speech. Qwen actually has a model and library for this: Qwen3-TTS [1]. [1]: https://github.com/QwenLM/Qwen3-TTS https://github.com/QwenLM/Qwen3-TTS
- deleted 8mo ago[deleted]
- deleted 8mo ago[deleted]
- benbojangles 8mo agoWas using Ollama but qwen3.5 unavailable earlier today
- codingbear 8mo agoDo they mention the hardware used for training? Last I heard there was a push to use Chinese silicon. No idea how ready it is for use
- deleted 8mo ago[deleted]
- fdefitte 8mo agoThe "native multimodal agents" framing is interesting. Everyone's focused on benchmark numbers but the real question is whether these models can actually hold context across multi-step tool use without losing the plot. That's where most open models still fall apart imo.
- nl 8mo ago"the post-training performance gains in Qwen3.5 primarily stem from our extensive scaling of virtually all RL tasks and environments we could conceive." I don't think anyone is surprised by this, but I think it's interesting that you still see people who claim the training objective of LLMs is next token prediction. The "Average Ranking vs Environment Scaling" graph below that is pretty confusing though! Took me a while to realize the Qwen points near the Y-axis were for Qwen 3, not Qwen 3.5.
- solarkraft 8mo agoPer my initial reading this thing is not only faster working with long context, but also very efficient storing it! Super excited for a ~30B version.
- inquirerGeneral 8mo ago[dead]