jychang

8 ms·

91.
by jychang 8mo ago
I am aware of all that. They literally never say “they used mxfp4 in some weights”. What you’re claiming they said doesn’t exist. This isn’t a postmortem, it’s PR fluff without actually addressing the issue.
92.
by jychang 8mo ago
32GB vram is more than enough for Qwen 3.5 35b You can just load the Q4_K_XL model like normal, and put all tensors on GPU without any -ot or --cpu-moe flags. If you need a massive context for some reason where model+kv cache won't fit
93.
by jychang 8mo ago
The Q4_K_XL is easily the most popular quant for the model, though. So then why was Q4_K_XL having issues? Is it just a PPL issue that doesn't reflect in real world usage? If yes, why not just say that? "The Q4_K_XL had lower PPL,
94.
by jychang 8mo ago
Yeah, I saw that yesterday. The blog post does not explain why/how the Qwen 3.5 quants uploaded on 2/27 are different from the files uploaded on 2/24. Old 2/24 Q4_K_XL commit (pre bugfix files): https://huggin
95.
by jychang 8mo ago
I'm aware of that, but that's not the link of the post. The post is linking to their UD 2.0 quants from a few months back. Also, the benchmarks are because they messed up the first version of their Qwen 3.5 XL quants by quanting s
96.
by jychang 8mo ago
What's up with this post? It's a link to something which has existed for a long time, and there's a bunch of dead comments below. Some weird SEO campaign thing?
97.
by jychang 8mo ago
Not really breakthroughs, more like bugfixes for their broken first batch.
98.
by jychang 8mo ago
It is that cheap. Look at Deepseek or GLM pricing.
99.
by jychang 8mo ago
That's fake epistemic humility, akin to a religious nutcase proclaiming "evolution is just a theory". In fact, he's using the exact same arguments. I'm not impressed. I've seen this before, from "biology i
100.
by jychang 8mo ago
> How hard would it be to systematically evaluate the different quantizations? E.g. on the Aider benchmark that you used in the past? Very hard. $$$ The benchmarks are not cheap to run. It'll cost a lot to run them for each quant of
101.
by jychang 8mo ago
This is more of an article describing their methodology than a full paper. But yes, there's plenty of peer reviewed papers on this topic, scaling sparse autoencoders to produce interpretable features for large models. There's a to
102.
by jychang 8mo ago
That's a lame typical anti-intellectual argument. You might as well as say all of physics is worthless because nobody truly understands gravity. Notice I didn't use vague terms like "think" or "reason" and inst
103.
by jychang 8mo ago
Lol that's a stupid ass response, especially when half the papers are from universities from China. You think the chinese universities are trying to sell ChatGPT subscriptions? Ridiculous. You're just falling behind in tech knowle
104.
by jychang 8mo ago
> I think nobody understands what's happening in these models Quick question, do you know what "Mechanistic Interpretability Researcher" means? Because that would be a fairly bold statement if you were aware of that. Try s
105.
by jychang 8mo ago
It's also literally factually incorrect. Pretty much the entire field of mechanistic interpretability would obviously point out that models have an internal definition of what a bug is. Here's the most approachable paper that show
106.
by jychang 8mo ago
Nice LLM generated text. Now go read https://transformer-circuits.pub/2024/scaling-monosemanticit... or https://arxiv.org/abs/2506.19382 to see why that text is outdated. Or read any paper in the
107.
by jychang 8mo ago
Ok, I'll bite. Let's assume a modern cutting edge model but even with fairly standard GQA attention, and something obviously bigger than just monosemantic features per neuron. Based on any reasonable mechanistic interpretability u
108.
by jychang 8mo ago
Nah, that's a cop out answer. Can't blame everything on shareholders when all the major Linux DEs do the same.
109.
by jychang 8mo ago
Yeah, it's pretty clear you're loud mouthed and don't know anything about distilling ML models or anything Cerebras. Distilling ML models into smaller mini versions is basic stuff. How do you think Qwen 3 235b and Qwen 3 30b
110.
by jychang 8mo ago
> I'd think a human being would be more likely to recognize they'd crossed a boundary with another human Please. We're autistic software engineers here, we totally don't do stuff like "recognize they'd cross
111.
by jychang 8mo ago
Yes, the difference is the latter means "it is no longer maintained", and the former is "they claim to be maintaining it but everyone knows it's not really being maintained".
112.
by jychang 8mo ago
Uh, that paragraph translated from "marketing bullshit" into "engineer" would be "we distilled the big gpt-5.3-codex model into a smaller size that fits on the 44GB of SRAM of a Cerebras WSE-3 multiplied by whatever
113.
by jychang 8mo ago
> If instead the model is performing worse due to how much they had to shrink it just so it will fit on Cerebras hardware They really should have just named it "gpt-5.3-codex-mini" (served by Cerebras). It would have made it cl
114.
by jychang 8mo ago
Yeah, that's like saying Romeo and Juliet by Shakespeare is obsolete because Romeo could have just sent Juliet a snapchat message. You're kinda missing the entire point of the story.
115.
by jychang 8mo ago
Well, it's more like they have 900,000 cores on a WSE and disable whatever ones that don't work. Seriously, that's literally just what they do.
116.
by jychang 8mo ago
It's buggy. Mission peak shows much of the bay area occluded.
117.
by jychang 8mo ago
What about the juice variable? https://www.reddit.com/r/OpenAI/comments/1qv77lq/chatgpt_low...
118.
by jychang 8mo ago
On the other hand, Deepseek V3.2 is $0.38 per million tokens output. And on openrouter, most providers serve it at 20 tokens/sec. At 20t/s over 1 month, that's... $19something running literally 24/7. In reality it'd
119.
by jychang 9mo ago
It was good for like, one month. Qwen3 30b dominated for half a year before that, and GLM-4.7 Flash 30b took over the crown soon after Nemotron 3 Nano came out. There was basically no time period for it to shine.
120.
by jychang 9mo ago
That seems like a nonissue for the purposes of this discussion though, in terms of user uptake. Tiktok and Facebook and other websites aren't exactly focused on serving to people on the same network.

More ›