4 ms·

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the lim

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

sealeck 4mo ago
Have we reached the limits of scaling? Sadly it appears that larger model still equals better model
- stogot 4mo ago
  It’s still diminishing returns yes? It isn’t Moore’s Law
- pixelready 4mo ago
  I think there’s still an open question around are the ultra-large next-gen models worth it? For those of us without early access to Mythos, it’s hard to verify whether it’s been held back from the public due to actually being “too dangerously powerful to release yet” as implied or because the gains aren’t outpacing the costs.
- mikestorrent 4mo ago
  Well, let's not forget that text models are not the only models! Video models are much slower and need comparatively more resources, and all they can do even at that size is generate videos a few seconds long. Clearly a ton more work is going to go into those, and demand for them will probably increase as more creative tools get authored using them as a central part of the workflow. Low-res local rendering for preview might be a thing, but the lion's share of the work for high-res, near-realtime rendering is going to be done on huge clusters for a long time yet.
  - niek_pas 4mo ago
    This is definitely a good point. I imagine the max capacity for video models is significantly lower than for text models (there just aren't as many professionals in video as there are people who write text or code) but I could be wrong.
- mindwok 4mo ago
  I think GPT 4.5 showed that there is indeed a practical limit we're close too. That was supposedly a high-trillions of parameter model that was deprecated almost immediately because it was slow, insanely expensive, and had questionable benefits over the smaller models. Though apparently the new Mythos and whatever GPT Spud is (if it wasn't 5.5) are back up in the high trillions.
  - XenophileJKO 4mo ago
    Actually having used it a bit, I'm quite excited to see a modern model of similar size. I think what people didn't realize was, just because the GPT-4.5 model didn't get better on the benchmarks, didn't mean the model wasn't different than the earlier models. It was being compared to thinking models that were being developed at the same time. The GPT 4.5 model still has some of the most "human" like abilities in communication even though it isn't particularly good a problem solving. It hadn't under gone the same type of reinforcement training. I still use GPT 4.5 sometimes, in creative exercises it can be surprisingly effective. The model is still available.
- adgjlsfhk1 4mo ago
  yes and no. We've reached the point where larger models are higher quality, but they're also too expensive and slow to be used broadly. The giant models, however are still useful for training smaller models that are actually deployable.
hadlock 4mo ago
Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that rate the ROI for local LLM is ~10 months
- strictnein 4mo ago
  Didn't Qwen stop releasing their more powerful models because they're commercializing them?
  - mswphd 4mo ago
    Yes and no. Qwen 3.5 was released 3/2/2026. It includes models up to a 397B-A17B model https://huggingface.co/collections/Qwen/qwen35 https://huggingface.co/collections/Qwen/qwen35 A day afterwards, a high-up technical leader working on Qwen was let go https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-steps-down-after-major-ai-push/ https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-st... The more recent Qwen 3.6 was released on 4/16 https://huggingface.co/collections/Qwen/qwen36 https://huggingface.co/collections/Qwen/qwen36 This does not include any particularly large models. But the models it contains (Qwen3.6 27B and Qwen3.6 35B-A3B) are the local models people have been very excited about lately. So they didn't release any larger models, and the models people praise so much are from this most recent release.
    - tyre 4mo ago
      If they stop releasing their larger models because they want to monetize, would we expect them to release better small models that can outcompete those?
      - mswphd 4mo ago
        there's pros and cons to it for them. Clearly, they get good branding (at least in enthusiast circles). Perhaps more important is they get community work on optimization. There have been significant performance uplifts on the Qwen3.6 models from the open-source community since they were launched (at a minimum, multi-token prediction is now working with them. It is almost a 2x token generation speedup) https://www.reddit.com/r/LocalLLM/comments/1ti9w4o/qwen3635ba3bmtp_on_an_rtx_3090_in_lm_studio_is/ https://www.reddit.com/r/LocalLLM/comments/1ti9w4o/qwen3635b...
cat5e 4mo ago
Huzzah, they’ve lost their stranglehold. Viva la revolution!
ActorNightly 4mo ago
Very false. I use small models exclusively. They aren't a replacement for large models. You need decent hardware to run those models efficiently, as smaller parameter models plain suck and are still slow on macbooks. And affordability of higher end hardware is very limited. Even at non VC subsidized $/token prices, its still much cheaper to run cloud based models.
- davnicwil 4mo ago
  well to be fair that's right now, I think the question is what about in 6 months, 12 months, 2 years? Where do these improvement curves go? Does the gap close, do they intersect for practical purposes (factoring in cost etc)? Or is the local curve always just a translation of the hosted, lagging behind, or indeed does hosted just pull ahead? Nobody knows, but it's a very open question I feel, and it certainly appears like the answer might quite reasonably be that yes they intersect on that kind of short-ish term time horizon.
  - ActorNightly 4mo ago
    >Where do these improvement curves go? Nowhere. Large models haven't seen that much improvement, just small unique tasks performance which is all special cased RLed to game metrics For local models, its the same story. You can download Gemma 3 QAT from last year, and it will be just as good as Gemma:31b on the average. Qwen also boasts that its better, because again, they RLed it to game some metrics. Its better in coding then Gemma, but Gemma is better in more creative thinking (again, all RL) Fundamentally, you need detail in the gradients for the models to pick up on the smaller details. If you don't have those, your output is gonna suck. No amount of clever architecture is going to fix this. The only way to improve local models by training them to fetch context, and then their job becomes much simpler because all they need to do is reinterpret the fetched content and provide an answer. But fundamentally, if you are trying to keep things in house for advertising purposes like what all companies do with search, you want them to go to your service, which means running on your servers. And its not really that much extra per invocation (i.e excluding initial hardware costs) to instead just offer a large model as a service, which will be way better than any small models.
viccis 4mo ago
I just want a tiny tiny model that runs on device that knows for autocomplete that, for example, I want to say "I'll be right back" instead of "I'll be right Brian". That's my #1 AI ask right now. Please, Apple.
- cush 4mo ago
  I want Siri to let me “add to my calendar, dinner Peter’s house Sunday at 5pm” and not assume the location is the restaurant called Peter’s House in another state. It’s astounding how poor Siri is at using the data I’ve given it access to
wyager 4mo ago
> By the end of next year you’ll be running most of your AI on device. I expect I'll probably keep paying for whatever badass high IQ model is running on inference servers at that point
zombiwoof 4mo ago
[dead]
maxdo 4mo ago
Why on earth I should switch from a top tier model to much worse local model ? Why do I need to suffer my battery ?
- truncate 4mo ago
  You can switch to local models for tasks/use-cases where you don't need top tier models.
- romanovcode 4mo ago
  Right now there is no reason since tokens are subsidized heavily. However when OpenAI/Anthropic will drop the $200/month pricing since most likely it eventually will become unsustainable you'd rather get MacBook Pro M6 Ultra with 128GB ram and go local then pay thousands every month for tokens.
hajile 4mo ago
In the coding realm, I think we'll be seeing 35, 70, and 150B models sold where you pay a few hundred to a few thousand dollars up front and get a year of monthly/bi-monthly updates where they've trained it on new coding documentation and repos.

sealeck 4mo ago

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

stogot 4mo ago
It’s still diminishing returns yes? It isn’t Moore’s Law
pixelready 4mo ago
I think there’s still an open question around are the ultra-large next-gen models worth it? For those of us without early access to Mythos, it’s hard to verify whether it’s been held back from the public due to actually being “too dangerously powerful to release yet” as implied or because the gains aren’t outpacing the costs.
mikestorrent 4mo ago
Well, let's not forget that text models are not the only models! Video models are much slower and need comparatively more resources, and all they can do even at that size is generate videos a few seconds long. Clearly a ton more work is going to go into those, and demand for them will probably increase as more creative tools get authored using them as a central part of the workflow. Low-res local rendering for preview might be a thing, but the lion's share of the work for high-res, near-realtime rendering is going to be done on huge clusters for a long time yet.
- niek_pas 4mo ago
  This is definitely a good point. I imagine the max capacity for video models is significantly lower than for text models (there just aren't as many professionals in video as there are people who write text or code) but I could be wrong.
mindwok 4mo ago
I think GPT 4.5 showed that there is indeed a practical limit we're close too. That was supposedly a high-trillions of parameter model that was deprecated almost immediately because it was slow, insanely expensive, and had questionable benefits over the smaller models. Though apparently the new Mythos and whatever GPT Spud is (if it wasn't 5.5) are back up in the high trillions.
- XenophileJKO 4mo ago
  Actually having used it a bit, I'm quite excited to see a modern model of similar size. I think what people didn't realize was, just because the GPT-4.5 model didn't get better on the benchmarks, didn't mean the model wasn't different than the earlier models. It was being compared to thinking models that were being developed at the same time. The GPT 4.5 model still has some of the most "human" like abilities in communication even though it isn't particularly good a problem solving. It hadn't under gone the same type of reinforcement training. I still use GPT 4.5 sometimes, in creative exercises it can be surprisingly effective. The model is still available.
adgjlsfhk1 4mo ago
yes and no. We've reached the point where larger models are higher quality, but they're also too expensive and slow to be used broadly. The giant models, however are still useful for training smaller models that are actually deployable.

hadlock 4mo ago

Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that rate the ROI for local LLM is ~10 months

strictnein 4mo ago
Didn't Qwen stop releasing their more powerful models because they're commercializing them?
- mswphd 4mo ago
  Yes and no. Qwen 3.5 was released 3/2/2026. It includes models up to a 397B-A17B model https://huggingface.co/collections/Qwen/qwen35 https://huggingface.co/collections/Qwen/qwen35 A day afterwards, a high-up technical leader working on Qwen was let go https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-steps-down-after-major-ai-push/ https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-st... The more recent Qwen 3.6 was released on 4/16 https://huggingface.co/collections/Qwen/qwen36 https://huggingface.co/collections/Qwen/qwen36 This does not include any particularly large models. But the models it contains (Qwen3.6 27B and Qwen3.6 35B-A3B) are the local models people have been very excited about lately. So they didn't release any larger models, and the models people praise so much are from this most recent release.
  - tyre 4mo ago
    If they stop releasing their larger models because they want to monetize, would we expect them to release better small models that can outcompete those?
    - mswphd 4mo ago
      there's pros and cons to it for them. Clearly, they get good branding (at least in enthusiast circles). Perhaps more important is they get community work on optimization. There have been significant performance uplifts on the Qwen3.6 models from the open-source community since they were launched (at a minimum, multi-token prediction is now working with them. It is almost a 2x token generation speedup) https://www.reddit.com/r/LocalLLM/comments/1ti9w4o/qwen3635ba3bmtp_on_an_rtx_3090_in_lm_studio_is/ https://www.reddit.com/r/LocalLLM/comments/1ti9w4o/qwen3635b...

cat5e 4mo ago

Huzzah, they’ve lost their stranglehold. Viva la revolution!

ActorNightly 4mo ago

Very false. I use small models exclusively. They aren't a replacement for large models. You need decent hardware to run those models efficiently, as smaller parameter models plain suck and are still slow on macbooks. And affordability of higher end hardware is very limited. Even at non VC subsidized $/token prices, its still much cheaper to run cloud based models.

davnicwil 4mo ago
well to be fair that's right now, I think the question is what about in 6 months, 12 months, 2 years? Where do these improvement curves go? Does the gap close, do they intersect for practical purposes (factoring in cost etc)? Or is the local curve always just a translation of the hosted, lagging behind, or indeed does hosted just pull ahead? Nobody knows, but it's a very open question I feel, and it certainly appears like the answer might quite reasonably be that yes they intersect on that kind of short-ish term time horizon.
- ActorNightly 4mo ago
  >Where do these improvement curves go? Nowhere. Large models haven't seen that much improvement, just small unique tasks performance which is all special cased RLed to game metrics For local models, its the same story. You can download Gemma 3 QAT from last year, and it will be just as good as Gemma:31b on the average. Qwen also boasts that its better, because again, they RLed it to game some metrics. Its better in coding then Gemma, but Gemma is better in more creative thinking (again, all RL) Fundamentally, you need detail in the gradients for the models to pick up on the smaller details. If you don't have those, your output is gonna suck. No amount of clever architecture is going to fix this. The only way to improve local models by training them to fetch context, and then their job becomes much simpler because all they need to do is reinterpret the fetched content and provide an answer. But fundamentally, if you are trying to keep things in house for advertising purposes like what all companies do with search, you want them to go to your service, which means running on your servers. And its not really that much extra per invocation (i.e excluding initial hardware costs) to instead just offer a large model as a service, which will be way better than any small models.

viccis 4mo ago

I just want a tiny tiny model that runs on device that knows for autocomplete that, for example, I want to say "I'll be right back" instead of "I'll be right Brian". That's my #1 AI ask right now. Please, Apple.

cush 4mo ago
I want Siri to let me “add to my calendar, dinner Peter’s house Sunday at 5pm” and not assume the location is the restaurant called Peter’s House in another state. It’s astounding how poor Siri is at using the data I’ve given it access to

wyager 4mo ago

> By the end of next year you’ll be running most of your AI on device. I expect I'll probably keep paying for whatever badass high IQ model is running on inference servers at that point

zombiwoof 4mo ago

[dead]

maxdo 4mo ago

Why on earth I should switch from a top tier model to much worse local model ? Why do I need to suffer my battery ?

truncate 4mo ago
You can switch to local models for tasks/use-cases where you don't need top tier models.
romanovcode 4mo ago
Right now there is no reason since tokens are subsidized heavily. However when OpenAI/Anthropic will drop the $200/month pricing since most likely it eventually will become unsustainable you'd rather get MacBook Pro M6 Ultra with 128GB ram and go local then pay thousands every month for tokens.

hajile 4mo ago

In the coding realm, I think we'll be seeing 35, 70, and 150B models sold where you pay a few hundred to a few thousand dollars up front and get a year of monthly/bi-monthly updates where they've trained it on new coding documentation and repos.