jychang

13 ms·

211.
by jychang 1y ago
So gpt-oss-20b is a sparse MoE model Which means it has ~3b parameters active per token. Qwen3-32b has 32b params active per token
212.
by jychang 1y ago
Exactly. You're ill equipped to answer the question because you don't know. Qwen is terrible at explaining what the difference is, between the models that they serve on their API. It's such a simple question: "For some
213.
by jychang 1y ago
If the naming is so clear to you, then why don't you explain: for a user who wants to use Qwen3-VL through an API, which one has better performance? Qwen3-VL Plus or Qwen3-VL 235b?
214.
by jychang 1y ago
They still suck at explaining which model they serve is which, though. They also released today Qwen3-VL Plus [1] today alongside Qwen3-VL 235B [2] and they don't tell us which one is better. Note that Qwen3-VL-Plus is a very different
215.
by jychang 1y ago
Yeah, this page seems to be not great for beginners and also useless for people with experience. A 2x 3090 build is okay for inference, but even with nvlink you're a bit handicapped for training. You're much better off with gettin
216.
by jychang 1y ago
You guys need a blog to chase these down. If nothing else, it’ll inspire the next generation of mathematicians.
217.
by jychang 1y ago
It's 1100 lines of code in a single file that nobody understands. That's instant spaghetti right there, not a valid PR. At least have it be split into some files and structured in some way.
218.
by jychang 1y ago
The human body consumes merely ~2000 calories a day. Really not that much energy, about the same as a 100W light bulb. Even if you can capture every single bit of extra energy from a tshirt, you'd end up with a tiny fraction of 100W. C
219.
by jychang 1y ago
90% of all important work is banal. That's kind of the thing. I'm sure a lot of engineering hours were spent on getting the door handle on your car to the exact safety/cost/functionality requirements, and at the end of t
220.
by jychang 1y ago
To clarify, I should have stated: "Instead of generating tokens one at a time, you generate the second one as well WITH MTP , and then use speculative decoding on that second token (instead of having the second token be produced by a
221.
by jychang 1y ago
> My second thought is how badly Chinese communism must be doing that they need such a massive effort in order to prevent their citizens from accessing information and voicing dissent. Well, OpenAI and other companies training AI models
222.
by jychang 1y ago
Strong disagree. Llama 4's release in 2025 is (deservedly) panned, but Llama 3.1 405b does not deserve that slander. https://artificialanalysis.ai/#frontier-language-model-intel... Do not compare 2024 models to the cur
223.
by jychang 1y ago
Because if you generate token n+1 with all 48 layers of Qwen3-Next and 80 billion params, and also generate token n+2 with the 1 MTP layer at 2bil params... that n+2 token can be much lower quality than the n+1 token but mostly correct. Let
224.
by jychang 1y ago
Speculative decoding! It makes inference a LOT faster. Instead of generating tokens one at a time, you generate the second one as well, and then use speculative decoding on that second token (instead of having it be produced by a draft mode
225.
by jychang 1y ago
Obviously yes, AI makes revenue.
226.
by jychang 1y ago
Coolest part of Qwen3-Next, in my opinion, (after the linear attention parts) is that they do MTP without adding another un-embedding matrix. Deepseek R1 also has a MTP layer (layer 61) https://huggingface.co/deepseek-ai&#x2
227.
by jychang 1y ago
None of that uses the ANE.
228.
by jychang 1y ago
To be fair, Apple HATES Nvidia after the 8400M and 9400M debacle. They probably saw replacing Nvidia as a bigger benefit than replacing Intel.
229.
by jychang 1y ago
Lol. I opened that link and was like "hmmm, that UI looks familiar". I'm pretty sure that's just a frontend for Uptime Kuma https://github.com/louislam/uptime-kuma
230.
by jychang 1y ago
Meh, your smartphone is already the ultimate spying device that comes with microphones and triangulates your location from 3 cell towers. The government doesn’t need more spyware than that.
231.
by jychang 1y ago
Yeah, I don't think most people who even run ollama would care. "Oh no, someone found my exposed instance, which means my computer in my bedroom is burning electricity for the past few hours. Oh well, I lost a few pennies in elect
232.
by jychang 1y ago
> is legally permitted to execute me. Plenty of cops have shot/executed innocent people, and were declared not guilty due to qualified immunity. Plus, a phone in China can't execute you either. It'd be the police goons who
233.
by jychang 1y ago
Yes, I agree here. Everyone should remember, limitations of technology is not meant to define society. Instead, we build edge cases into technology to better match society’s general expectations. A website owner saying “yes normal humans, n
234.
by jychang 1y ago
> What's the value of "pointing out limitations" if this completely fails to drive any improvements? Ironically, the same could be said about Attention Is All You Need in 2017. It didn’t drive any improvements immediately-
235.
by jychang 1y ago
https://en.m.wikipedia.org/wiki/Artificial_neuron
236.
by jychang 1y ago
Do you also disagree with the use of the name “tree” in a computer science class? Again, nobody thinks trees in computer science contains squirrels, nobody thinks airplanes are birds, and nobody thinks a neuron in a ML model contains axons
237.
by jychang 1y ago
1. They are neurons, whether you like it or not. A binary tree may not have squirrels living in them, but it's still a tree, even though the word "tree" here is defined differently than from biology. Or are you going to say a
238.
by jychang 1y ago
I dislike the term “stochastic parrot”, because there’s plenty of evidence that LLMs do have an understanding of at least some things that they are saying. We can trace which neurons activate for a face recognition model and see that a cert
239.
by jychang 1y ago
The biggest problem with Debian 13 is not with Debian, it's with people like Google and Cloudflare. Come on guys, Debian 13 has been in testing for months, and you can't be arsed to update your apt repos from bookworm to trixie by
240.
by jychang 1y ago
That hasn't worked for china since before 2020, you're years out of date.

More ›