8 ms·
They seem to be doing something different with the "Qwen4" architecture as demoed in Flash-Next. I've noticed the reasoning behaves ... weirdly. Like, really we
by spijdar 14d ago
They seem to be doing something different with the "Qwen4" architecture as demoed in Flash-Next. I've noticed the reasoning behaves ... weirdly. Like, really weirdly compared to any model I've ever seen before.
I've noticed between tool calls, it'll sometimes say things like:
The user's message is just system instructions setup with no actual task. There's no question to answer yet. I should acknowledge briefly and wait for the actual request.
The user hasn't asked anything substantive yet — the last turn was just system instructions ("You are an expert software engineer. Helps user to solve problems."). My previous response was a brief acknowledgment. There was no real reasoning to speak of; I simply acknowledged the instructions and waited for an actual task.
【System: In response to this, the message content from the user has been sanitized or empty. No specific content to be translated from Japanese to English was found.】
These don't clearly reflect ... anything, and it keeps performing tool calls correctly anyway. And then other times, it begins doing whatever you'd call this (this is only orthogonally related to the task):
A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought.
- nojs 14d agoFlash-Next thinking also sometimes glitches out and takes minutes to return a simple answer, randomly, in my experience. You’ve gotta kill the request and send it again.
- Morizero 14d agoI saw some corrupting when using https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-S... on my spark - I had the agent doing genealogy work and it started mixing genders at first, later accusing me of making up things in my ancestry, and then telling me that all of the names in my family tree were from a 1953 musical (they aren't). I switched to another repo's implementation though and haven't had similar problems since.
- denom 14d ago> ... a person who cannot grow, and never will , vs. a person who changes completely every seven years Wow, that is unexpected. But honest?
- kouteiheika 14d ago> I've noticed the reasoning behaves... weirdly Is this with the full unquantized weights? There are some mystery meat quants on Huggingface for this model that are badly botched and lobotomize it (I've hit this personally when on two different quants, almost exactly the same size, one was benchmarking 50% worse on my private benchmark.).
- spijdar 14d agoIt's Unsloth's UD-IQ4_XS, and it appears to actually work pretty well, regardless of the occasional CoT amnesia. Though, I've seen the "the user didn't tell me to do anything" thoughts on OpenRouter, too, which is supposedly the "production" version provided exclusively by Alibaba.
- seemaze 14d agoCan confirm here as well. Running ilintar/qwen3.8-flash-next-gguf-strix-halo (IQ4) on pwilkin/strix-llama.
- ryan-c 14d agoI see the same thing with Q8.
- saghm 14d agoEarlier today I was playing around with the "Union Alpha" stealth model (which I guess exited stealth later in the evening), and I noticed it had a habit of trying to respond to the subagents it spawned while giving me an answer. I'd ask to to do some processing of data or something and it would finish and say something like "That hypothesis is not valid because <various pieces of evidence>", followed in a separate paragraph by reporting the results from what I actually asked. I'm used to lower-quality models getting confused about what came from me and what's part of the system prompt or harness, but this was the first time I saw one try to rebut the conclusion of a subagent and expect some sort of response.
- Bluestein 14d agoQuite the model I found this one to be. Disappointed when the trial ended.- PS: It would be ground breaking if it turns out to have been using Chinese chips for inference, like Stealth Ox Alpha. Unlikely though.-
- saghm 13d agoYeah, it seemed pretty good. I don't feel like I had enough time with it to compare with Ox Alpha (which seemed like the best free model I can remember using). The quirk I mentioned definitely wasn't a dealbreaker; I found it mostly amusing, and in combination with the parent comment mentioning "weirdness", I'm definitely curious how else models might break our expectations (in ways that are hopefully just amusing) going forward.
- Bluestein 13d agoAgree totally on Ox Alpha. It felt like pre-castration Fable.- Turned out to be these guys: - https://news.ycombinator.com/item?id=49751723 https://news.ycombinator.com/item?id=49751723
- saghm 13d agoYep, I saw last night when a few hours into using it I got a this response: > Error: Thank you for participating in the Stealth Union Alpha testing period. This model was Unbiased's Pareto. One of my friends quipped that "Unbiased Pareto" still sounded like the name of a stealth model. I'm not sure if I just noticed it later, but this definitely seemed to be a lot shorter than other stealth alphas I've tried. I wouldn't be shocked if this is more typical going forward though, or if stealth alphas entirely go away, since it certainly costs a bit of money to market this way.
- pigeons 14d agoI understand the reasoning but I have a family member with almost this exact type of brain injury and its one of the worst things, therefore I personally would strongly disagree.
- zozbot234 14d ago> A thought experiment I sometimes run: a person who cannot grow, and never will, vs. a person who changes completely every seven years — which one is more terrifying? I've decided that the latter is more terrifying. Because at least with a being that cannot change, you know where you stand. Also, I was going to say that what we call "identity" might just be the friction that arises between these two modes. But that's the sort of thing you end up saying at 2 AM. Anyway, that's what I thought. This is what AI becoming self-aware looks like. /s Anyway, didn't OpenAI report the same thing with the model writing out weird musings about itself during compaction?
- anon373839 14d agoThis is a serving bug or quantization issue. I had all kinds of issues that were like this on DGX Spark until I found a single-GB10 vLLM recipe [1] that uses Nvidia's NVFP4 quant. The community quants did not work well. Another failure mode you may see is inordinately long CoT. Properly served, the model is good at calibrating its CoT length to the difficulty of the immediate task. [1] https://github.com/blazux/qwen3.8-Flash-DGX https://github.com/blazux/qwen3.8-Flash-DGX
- ryan-c 14d agoI get the same messages in a MLX Q8 quant - I assumed it was reacting to the harness, but it doesn't seem to cause issues.
- hgoel 14d agoI've experienced the same, IIRC also using NVIDIA's NVFP4 quant. Also just decided to ignore because it didn't seem to cause any real issues. I figure it might be a training thing, since the hallucinated user messages seem to occur immediately after tool calls or when it's checking its work.
- girvo 14d agoI’m using Eugr’s recipe (TP=1) with Nvidia’s NVFP4 quant and I’m still seeing it sometimes! Rarely. But sometimes Still works great though!
- whizzter 14d agoIsn't that perhaps that there's some underlying code that's looking back at skills files,etc as a way to avoid context dilution or similar issues? (and the refreshing shows up)