6 ms·

If this is true, the hyperscalers are toast

52 points by root-parent 2mo ago

palata 2mo ago
"If", sure. How many developers here don't see a difference between the latest LLMs and SLMs they can run on their own computer? I tried running a smaller model locally, and it's not usable for me. I know people like to "predict" things, so that if they happen they can then say "I am a visionary, I predicted it" and start their blog posts with "as I predicted long ago (because I am a visionary), ...". > The research report estimates that the addressable market in the US for SLMs has grown to about $10tn or one-third of the entire US GDP of $30tn. There isn’t much left for LLMs to thrive in, and every year, their advantage over SLMs is shrinking. I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock. If something better comes, it will be better. Sure. And we would like to have something better, because it would be better.
- otabdeveloper4 2mo ago
  > I tried running a smaller model locally, and it's not usable for me. Probably a skill issue on your part.
- root-parent 2mo ago
  >> I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock. The lack of logic and risk management on this statement, is so strong, I hope humans are all quickly substituted by LLMs. Lets just do it and be done with it...
  - tialaramex 2mo ago
    > Lets just do it and be done with it... Presumably not what you intended but this phrase immediately takes me to: https://www.youtube.com/watch?v=dJFR7xbOIuw&t=42s https://www.youtube.com/watch?v=dJFR7xbOIuw&t=42s
    - root-parent 2mo ago
      Great movie...yeah I think I was inspired by the scene... :-)
  - palata 2mo ago
    > I hope humans are all quickly substituted by LLMs Why don't you go talk to your LLM instead of commenting here, then?
- ch_sm 2mo ago
  > I tried running a smaller model locally, and it's not usable for me. If you have the hardware, a MacBook Pro for Qwen 3.6 35B A3B and Gemma 4 26B A4B for example, they are absolutely usable, both in terms of speed and quality. Anecdotally, I can use Qwen for day-to-day coding tasks in TS and Go, without hickups.
  - everyone 2mo ago
    You let a hiccup slip through in your comment though.
    - JKCalhoun 2mo ago
      I live in fly-over country and I am offended by their use of "hickup".
    - ch_sm 2mo ago
      prooof-read by Qwen
  - embedding-shape 2mo ago
    I'm unable to find a local model that comes close to the effectiveness of GPT models in Codex, and I have 96GB of VRAM available and tried every local model under the sun so far. Neither of those you mention I'd say are good enough for day to day software engineering for me, but I'm also really strict about code quality and iterate on what outputs agents give me a lot before I'm happy. With local models, this iteration cycle takes maybe 30 minutes for a single fix or feature, rather than 10 minutes with GPT+Codex, as there is so many corrections and iterations needed, although I will say that the speed I'm able to get locally makes it more fun that any of the remote models.
    - rapind 2mo ago
      > although I will say that the speed I'm able to get locally makes it more fun that any of the remote models. This is becoming increasingly important to me. Super smart max reasoning frontier is fine if I leave it running overnight on some prepared set of clearly defined tasks, but when I want to work with the LLM, throughput really matters, and I'll go with a dumber model to get there. At some point though, it's fast enough and any speed gains beyond that just makes me the bottleneck. I also am seeing the smaller models gaining big strides lately, closing the gap on frontier models (still a decent sized gap though). I don't even run the small models like Qwen 3.8 27B locally. I just try them out in the cloud to see how they are progressing, and I'm definitely able to be productive.
- deleted 2mo ago
  [deleted]
nubg 2mo ago
As much as I want local and open-weights models to succeed, nothing beats a paid frontier model for now. Anybody who claims otherwise is simply not a daily user of such models. So this "investor" here should invest sime time in actually using the various LLM models and get a real taste of what it's like.
- trescenzi 2mo ago
  Their point isn’t that local models are better or even as good more but that if you can do 50%+ of tasks with local then that’s 50% of tokens that aren’t captured as compute done in data centers.
  - popularonion 2mo ago
    > As you can see, on average, SLMs are as good if not better than LLMs in 81.2% of the cases, with the LLMs having a significant advantage only in areas like engineering, life sciences, transportation and computer sciences. So what I’m reading here is “LLMs have a significant advantage” in the most critical areas that have practically infinite demand for more intelligence.
    - eigenspace 2mo ago
      The article is kinda dumb, and yes this is clearly the area where frontier models having and advantage matters the most, but I'd point out that these smaller open-weight models are performing better than the big Frontier models of just 4-6 months ago. This means that the Frontier labs are under immense pressure to maintain that lead, and could end up in serious trouble if they stumble at all. The other thing id point out is that a lot of us who are token-sensitive do things like build plans using expensive, smart models, and then execute those plans using cheaper dumber models. Then there's the fact that we are still in the age of heavily subsidized Frontier subscriptions + tokenmaxxing initiatives from megacorps. Neither of which are sustainable, and will drive more usage to smaller open models once they end.
- root-parent 2mo ago
  You completely missed the thesis here, and that is supported by the numbers being presented. It is that a large share of ordinary inference can be routed away from the hyperscalers.
CTDOCodebases 2mo ago
Haven't the SLMs been distilled using the LLMs? If this is correct I see a future where the hyperscalers are funded by the businesses integrating siloed SLMs in their software. Also the defence/intelligence industry will always want to keep an edge so don't be surprised if they stick around and we see favourable regulations for them similarly to how the government turns a blind eye to social media platforms because they increase the footprint of mass surveillance. I wouldn't be surprised if the hyperscalers became software auditors and any piece of critical software was required to have a regulated security audit before it could enter production. Selling the poison and the cure is a great business model.
Animats 2mo ago
A remaining advantage of large language models is that as they get larger, they tend to hallucinate less, simply because the odds of the training set containing a desired answer improve with size. If a solid "I don't know" detector is developed for inference, then you can try a small language model first. An implication is that successful research in "I don't know" detection could destroy hundreds of billions in shareholder value.
- embedding-shape 2mo ago
  Another "cool but we don't know how yet" thing would be a "confidence interval" so we know how much to trust LLM responses. Or while we're fantasizing, they could just know everything all the time regardless of training data. The "if a solid" part is easy to imagine, hard to implement :)
- root-parent 2mo ago
  >> A remaining advantage of large language models is that as they get larger, they tend to hallucinate less First time I hear that...not really true. "Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors" - https://arxiv.org/abs/2607.00447 https://arxiv.org/abs/2607.00447 "Calibrated Language Models Must Hallucinate" - https://arxiv.org/abs/2311.14648 https://arxiv.org/abs/2311.14648 "TruthfulQA: Measuring How Models Mimic Human Falsehoods" - https://arxiv.org/abs/2109.07958 https://arxiv.org/abs/2109.07958
  - Animats 2mo ago
    From the "must hallucinate" paper: "For "arbitrary" facts whose veracity cannot be determined from the training data, we show that hallucinations must occur at a certain rate for language models that satisfy a statistical calibration condition appropriate for generative language models." The bigger the model, the more likely it is that arbitrary facts in the training data are embedded in the model. Then a larger model shows less hallucination on the same questions, since it has a matching answer stored for more questions. From the "TruthfulQA" paper: "Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution." That's more of a garbage-in, garbage out problem. If the large model is trained by shoveling in random web content, that's going to happen. Not a hallucination problem. The LLM just fed back what it had been told.
cucumber3732842 2mo ago
Cool, they scored well on all the "make complex calculations and I'll vibe check your results based on my own domain experience" things I use the average LLM chatbot for. So maybe in 10yr I'll be able to run a SLM on a 5yo laptop and not have Google or whoever hoover up everything.
hyperhello 2mo ago
> If their results are true, then we will hardly need any data centres in the future, and the hyperscalers are wasting hundreds of billions of dollars in investments. What if they get sufficiently powered and watered industrial warehouses close to where the successful people live?
throwthrowuknow 2mo ago
From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth. Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? They’ll likely be shipped as an ASIC (or MSIC) at that point anyways. Those will use a licensed model from the current leaders. The whole argument sounds like saying that cloud services shouldn’t be profitable because everyone has a computer at home or to meme “we have AI at home”.
- mcphage 2mo ago
  > Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? Apple or Nvidia, presumably.
  - throwthrowuknow 2mo ago
    Hardware yes, weights? lol
    - mcphage 2mo ago
      Why not? Apple would have you download an updated set of weights with OSX / iOS updates. Nvidia could bundle them with driver updates. I don't think either would struggle to get people to keep their weights up to date.
      - throwthrowuknow 2mo ago
        I’m sure it would be fine for simple routine tasks but it wouldn’t compete with OpenAI and Anthropic models
        2mo ago
philipallstar 2mo ago
This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.
- embedding-shape 2mo ago
  You have to remember that articles like these are written for finance people who don't understand the underlying technology, by finance people who don't understand the underlying technology. In this case, the author is a "CFA Institute Enterprising Investor", previously a CIO and basically their entire life been "money, money & money", so hardly surprising they're pulling a lot of assumptions based on what they read. Read the paper the author talks about yourself instead (https://arxiv.org/abs/2511.07885 https://arxiv.org/abs/2511.07885), and also, contrary to what the author says in the article, do not do investments based on single papers made from academic studies, regardless of how much money this guy tells you you can make.
- amelius 2mo ago
  The logic seems mad to me because SLMs can simply not hold as much information as an LLM. Maybe if you combine an SLM with a database (as a tool) then it could work, but someone should first prove that.
  - fph 2mo ago
    But do you really need a model that has the complete Duran Duran discography memorized and preloaded in RAM at all time?
    - amelius 2mo ago
      That's a different question. Probably not. But: 1. Training a large model with lots of information, then stripping the "useless" information from that model to obtain a small model => nobody has shown this. 2. Training a small model, letting it use a database tool so it scores the same as a large model without database => nobody has shown this.
Zigurd 2mo ago
If you are like Google or Apple and you are delivering AI to a mass market unwilling to pay a lot for it, you are absolutely going to drive AI processing to endpoint devices. You are also going to spend what it takes in R&D make a hybrid system that knows when to use local compute or cloud compute. That's going to be the bulk of the workload.
simonebrunozzi 2mo ago
The paper focuses on "intelligence per watt (IPW)", as a way to compare SLMs vs LLMs. What might happen is that a chunk of the market, whatever its size will be, will end up going to SLMs run on iphones or Macbooks, and eat some of the revenues from LLMs, because not everyone needs the most powerful LLM all the time.
eddie_catflap 2mo ago
This is up to October 2025 though, yes? Obviously things are continually moving but Opus 4.5 launched in November and that was a recognised step change in capability. An up to date comparison would be interesting.
Havoc 2mo ago
Complete nonsense. > they provide a better or at least as good an answer as LLMs in 62.5% of the cases. Are we going to scrap hospitals because a vet could do the job 62.5% of the time? The economics also point away from everyone buying a big RAM Mac that sits idle 99% of the time. SLM and own hardware sounds efficient and “free” but it is nothing of the sort when you factor everything in (and forfeit the sharing efficiencies of API) SLMs are great esp for task specific fine tunes but this take isn’t it
physicsguy 2mo ago
One of the big things to think about is whether local LLMs will be things companies want to deploy. If you think of for e.g. some proprietary piece of software that wants to embed an LLM they've fine tuned or trained, they will want to make back some of their research cost right. So they are not going to want to put this on-device even if the hardware is there, unless there's some way of locking it down. I suspect we'll need on-hardware validation/verification and a way of preventing extraction of weights for this move to happen for many use cases.
- esseph 2mo ago
  > One of the big things to think about is whether local LLMs will be things companies want to deploy. Non-tech enterprise was already doing this years ago. Regulatory reasons, privacy reasons, security, etc. They want on-prem and total ownership of the data. Sometimes air-gapped.
  - roryirvine 2mo ago
    Honestly, I've seen a lot less of this than I'd have expected. Lots of discussion, plenty of experiments or toy implementations at the level of individual teams (often driven by one or two enthusiastic individuals), a few proofs of concept for internal services at a more strategic level, but pretty much nothing beyond that. This is in the UK where there's currently a big focus on data sovereignty in general, and I'm genuinely surprised by how little that's spilled over into demand for inference sovereignty (so far). I still expect demand to grow substantially, but I've been saying that for the past couple of years and am beginning to wonder if there'll need to be some sort of trigger event before it happens (eg. the datacentre bubble bursting, or some sort of major scandal).
js8 2mo ago
I believe it is true, and likely there exists a class of even smaller models than what they call "small". You can imagine a reasoning model as a huge set of rules that generate the next statement from previous statements (written in context). In that sense, a reasoning model can be compared to a logical theory - you have certain deduction rules which can generate new judgments. Often, logical theories are structured that the rules are remade into axioms, and the deduction rule is only modus ponens (which corresponds to function application and is a building block of program execution). In the case of an LLM, the set of rules (or axioms) they have in the theory is quite large, but most likely semantically unsound (with respect to their their own representation of truth) - that's why LLM's make mistakes. It would be desirable to break the logical theory represented by LLM into a smaller set of axioms, which would: a) remove rules easily deductible from the smaller core of axioms (for example, LLM doesn't need to remember "Socrates is mortal", as it can derive it from "Socrates is a man" and "all men are mortal") b) remove rules that have low value (facts that aren't used often or have weak validity) which cause ruleset to become unsound I suspect that's what SLM distillation is doing, to some extent. The question is, how far this process can go? I personally believe there is a useful logic for commonsense reasoning that has less than thousand rules (still several orders more than your typical mathematical logic, but orders less than SLMs). These axioms do not contain much facts about the world, but that could be added. So I believe there is a sweet spot (deductive core, encyclopedic shell) which we have not yet found (it's a little bit more formal language than natural language) but is very efficient for general reasoning.
kyleblarson 2mo ago
Given how often the "experts" on CNBC and Bloomberg TV use the term hyperscalers, my approach is to completely disregard anything written or spoken by any person who uses the term.
- beepbooptheory 2mo ago
  What would be a better term?
  - root-parent 2mo ago
    It has to be the Hyperspenders
- root-parent 2mo ago
  Two weeks ago, CNBC invited one of their "experts" who focus on SpaceX, and he said they have 10 million satellites in orbit. This is the current level of financial journalism available on "specialized" financial channels...
aslkalska 2mo ago
I don't think they are toast, I mean they will be in some trouble because all of them have fallen victim to fomo and started building out with so much debt for capacity that may or may not be needed nor achieve the returns that they want. I think there's a future where "personal software" meaning highly custom apps generated by an agent is a thing that doesn't mean everything will become that, same for local LLMs but all of this is still too far. The main issue is that hyperscalers or big tech in general have become too powerful they can just buy their way in and out of legislation as they please, sorry I mean lobby ... funny how if you rename something it becomes legal or illegal
- spinningslate 2mo ago
  Exactly. Seems naive at best for an investment consultant to look narrowly at current model capability and not consider the broader market. For example: 1. The hyperscalers are in a positive reinforcement loop. Despite any suggestions to the contrary they keep getting bigger. And can, er, “influence” government policy/officials and anything else needed to keep it that way. 2. The frontier labs and their investors. Another self-fulfilling reinforcement loop. Witness the circular gymnastics among OAI/Anthropic, Microsoft/Amazon and Nvidia 3. Data. No-one believes that Zuckerberg and co are going to say “great, we can just run the models on devices we don’t own and stop the surveillance economy because, y’know, privacy matters and we really care about mental health”. And then there’s data centre locations and “yeah but jobs” even though your power bills are going up, and “why run your own data centre Mrs CTO, let us do it for you and save all that capex and those pesky employees you need to do it”. Don’t get me wrong: I’m rooting for local, open weight/source models. But “hey look they benchmark well” is an unhelpfully narrow basis to forecast the demise of central hyperscaler hosting.
  - aslkalska 2mo ago
    oh don't get me started on finance "bros" - they just normalized greed - and that will probably be the doom of us all ... yeah I don't really know what people are doing with these local models - I tried the latest qwen3.8 27b on my decent consumer grade GPU and it struggles to do anything useful when I'm pretty sure Claude would have probably finished the same thing in a couple of hours, like it's not even close.
andai 2mo ago
There's also video models, world models, robotics simulations, the matrix...
andai 2mo ago
Small language model gave satisfactory healthcare output in 100% of cases?
pu_pe 2mo ago
The paper underlying this blog post is fundamentally flawed because of benchmark ceilings. If we define only simple tasks like asking what is the capital of France, all models will converge to 100%, obviously. But as bigger models get more capable we want them to replace more and more complex tasks, in as short time as possible. Then of course there is the economics of it. Do people prefer to spend $5000 upfront to get things done 5x slower, or would they rather pay $20 a month for that?
- conorcleary 2mo ago
  They'll start labeling scale using panamax-like terminology once they've met and overcome moore's law of this or that, comparing things that hadn't been conceived of yet against an easily memberable/quotable rule.
Garlef 2mo ago
I think one of the watershed developments will be fast models. Imagine current frontier models at 20k tokens/second.
- JKCalhoun 2mo ago
  I can't read that fast. Maybe this is interesting for an agent?
Jamesbeam 2mo ago
I think it’s a bit more complicated. There is a likely US scenario and a rest of the world scenario. It will be interesting to see if China acts on the overextension of the US Military in the Middle East. Taiwan will be a big play for both and crucial to the hyperscalers. But since the US is dabbling in piracy again and telling people what they can do and not do with their shit, it’s not too far-fetched that everyone that is not a global superpower is at risk of getting bombed to smithereens if they are a danger to US AI supremacy. This is such a crazy timeline, predicting even like a single year ahead feels like looking into a medieval glass ball. But we are humans, I am confident we will find a way to fuck this up royally for everyone. Brace for impact.
1vuio0pswjnm7 2mo ago
"The future could well be specialized edge models that know only ONE thing - and know it well." That design paradigm sounds familiar Everyday I use smalll applications that do only one thing, some written in the 1970's This submission got [flagged]. Later the "[flagged]" label was removed
- JKCalhoun 2mo ago
  "That design paradigm sounds familiar" In fact sounds like an expert system from decades ago.

52 points by root-parent 2mo ago

"If", sure. How many developers here don't see a difference between the latest LLMs and SLMs they can run on their own computer? I tried running a smaller model locally, and it's not usable for me. I know people like to "predict" things, so that if they happen they can then say "I am a visionary, I predicted it" and start their blog posts with "as I predicted long ago (because I am a visionary), ...". > The research report estimates that the addressable market in the US for SLMs has grown to about $10tn or one-third of the entire US GDP of $30tn. There isn’t much left for LLMs to thrive in, and every year, their advantage over SLMs is shrinking. I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock. If something better comes, it will be better. Sure. And we would like to have something better, because it would be better.

otabdeveloper4 2mo ago
> I tried running a smaller model locally, and it's not usable for me. Probably a skill issue on your part.
root-parent 2mo ago
>> I stopped counting the number of times "estimates" said that a market would absolutely explode, and it absolutely didn't. Those are in the business of being a broken clock. The lack of logic and risk management on this statement, is so strong, I hope humans are all quickly substituted by LLMs. Lets just do it and be done with it...
- tialaramex 2mo ago
  > Lets just do it and be done with it... Presumably not what you intended but this phrase immediately takes me to: https://www.youtube.com/watch?v=dJFR7xbOIuw&t=42s https://www.youtube.com/watch?v=dJFR7xbOIuw&t=42s
  - root-parent 2mo ago
    Great movie...yeah I think I was inspired by the scene... :-)
- palata 2mo ago
  > I hope humans are all quickly substituted by LLMs Why don't you go talk to your LLM instead of commenting here, then?
ch_sm 2mo ago
> I tried running a smaller model locally, and it's not usable for me. If you have the hardware, a MacBook Pro for Qwen 3.6 35B A3B and Gemma 4 26B A4B for example, they are absolutely usable, both in terms of speed and quality. Anecdotally, I can use Qwen for day-to-day coding tasks in TS and Go, without hickups.
- everyone 2mo ago
  You let a hiccup slip through in your comment though.
  - JKCalhoun 2mo ago
    I live in fly-over country and I am offended by their use of "hickup".
  - ch_sm 2mo ago
    prooof-read by Qwen
- embedding-shape 2mo ago
  I'm unable to find a local model that comes close to the effectiveness of GPT models in Codex, and I have 96GB of VRAM available and tried every local model under the sun so far. Neither of those you mention I'd say are good enough for day to day software engineering for me, but I'm also really strict about code quality and iterate on what outputs agents give me a lot before I'm happy. With local models, this iteration cycle takes maybe 30 minutes for a single fix or feature, rather than 10 minutes with GPT+Codex, as there is so many corrections and iterations needed, although I will say that the speed I'm able to get locally makes it more fun that any of the remote models.
  - rapind 2mo ago
    > although I will say that the speed I'm able to get locally makes it more fun that any of the remote models. This is becoming increasingly important to me. Super smart max reasoning frontier is fine if I leave it running overnight on some prepared set of clearly defined tasks, but when I want to work with the LLM, throughput really matters, and I'll go with a dumber model to get there. At some point though, it's fast enough and any speed gains beyond that just makes me the bottleneck. I also am seeing the smaller models gaining big strides lately, closing the gap on frontier models (still a decent sized gap though). I don't even run the small models like Qwen 3.8 27B locally. I just try them out in the cloud to see how they are progressing, and I'm definitely able to be productive.
deleted 2mo ago
[deleted]

nubg 2mo ago

As much as I want local and open-weights models to succeed, nothing beats a paid frontier model for now. Anybody who claims otherwise is simply not a daily user of such models. So this "investor" here should invest sime time in actually using the various LLM models and get a real taste of what it's like.

trescenzi 2mo ago
Their point isn’t that local models are better or even as good more but that if you can do 50%+ of tasks with local then that’s 50% of tokens that aren’t captured as compute done in data centers.
- popularonion 2mo ago
  > As you can see, on average, SLMs are as good if not better than LLMs in 81.2% of the cases, with the LLMs having a significant advantage only in areas like engineering, life sciences, transportation and computer sciences. So what I’m reading here is “LLMs have a significant advantage” in the most critical areas that have practically infinite demand for more intelligence.
  - eigenspace 2mo ago
    The article is kinda dumb, and yes this is clearly the area where frontier models having and advantage matters the most, but I'd point out that these smaller open-weight models are performing better than the big Frontier models of just 4-6 months ago. This means that the Frontier labs are under immense pressure to maintain that lead, and could end up in serious trouble if they stumble at all. The other thing id point out is that a lot of us who are token-sensitive do things like build plans using expensive, smart models, and then execute those plans using cheaper dumber models. Then there's the fact that we are still in the age of heavily subsidized Frontier subscriptions + tokenmaxxing initiatives from megacorps. Neither of which are sustainable, and will drive more usage to smaller open models once they end.
root-parent 2mo ago
You completely missed the thesis here, and that is supported by the numbers being presented. It is that a large share of ordinary inference can be routed away from the hyperscalers.

CTDOCodebases 2mo ago

Haven't the SLMs been distilled using the LLMs? If this is correct I see a future where the hyperscalers are funded by the businesses integrating siloed SLMs in their software. Also the defence/intelligence industry will always want to keep an edge so don't be surprised if they stick around and we see favourable regulations for them similarly to how the government turns a blind eye to social media platforms because they increase the footprint of mass surveillance. I wouldn't be surprised if the hyperscalers became software auditors and any piece of critical software was required to have a regulated security audit before it could enter production. Selling the poison and the cure is a great business model.

Animats 2mo ago

A remaining advantage of large language models is that as they get larger, they tend to hallucinate less, simply because the odds of the training set containing a desired answer improve with size. If a solid "I don't know" detector is developed for inference, then you can try a small language model first. An implication is that successful research in "I don't know" detection could destroy hundreds of billions in shareholder value.

embedding-shape 2mo ago
Another "cool but we don't know how yet" thing would be a "confidence interval" so we know how much to trust LLM responses. Or while we're fantasizing, they could just know everything all the time regardless of training data. The "if a solid" part is easy to imagine, hard to implement :)
root-parent 2mo ago
>> A remaining advantage of large language models is that as they get larger, they tend to hallucinate less First time I hear that...not really true. "Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors" - https://arxiv.org/abs/2607.00447 https://arxiv.org/abs/2607.00447 "Calibrated Language Models Must Hallucinate" - https://arxiv.org/abs/2311.14648 https://arxiv.org/abs/2311.14648 "TruthfulQA: Measuring How Models Mimic Human Falsehoods" - https://arxiv.org/abs/2109.07958 https://arxiv.org/abs/2109.07958
- Animats 2mo ago
  From the "must hallucinate" paper: "For "arbitrary" facts whose veracity cannot be determined from the training data, we show that hallucinations must occur at a certain rate for language models that satisfy a statistical calibration condition appropriate for generative language models." The bigger the model, the more likely it is that arbitrary facts in the training data are embedded in the model. Then a larger model shows less hallucination on the same questions, since it has a matching answer stored for more questions. From the "TruthfulQA" paper: "Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution." That's more of a garbage-in, garbage out problem. If the large model is trained by shoveling in random web content, that's going to happen. Not a hallucination problem. The LLM just fed back what it had been told.

cucumber3732842 2mo ago

Cool, they scored well on all the "make complex calculations and I'll vibe check your results based on my own domain experience" things I use the average LLM chatbot for. So maybe in 10yr I'll be able to run a SLM on a 5yo laptop and not have Google or whoever hoover up everything.

hyperhello 2mo ago

> If their results are true, then we will hardly need any data centres in the future, and the hyperscalers are wasting hundreds of billions of dollars in investments. What if they get sufficiently powered and watered industrial warehouses close to where the successful people live?

throwthrowuknow 2mo ago

From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth. Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? They’ll likely be shipped as an ASIC (or MSIC) at that point anyways. Those will use a licensed model from the current leaders. The whole argument sounds like saying that cloud services shouldn’t be profitable because everyone has a computer at home or to meme “we have AI at home”.

mcphage 2mo ago
> Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? Apple or Nvidia, presumably.
- throwthrowuknow 2mo ago
  Hardware yes, weights? lol
  - mcphage 2mo ago
    Why not? Apple would have you download an updated set of weights with OSX / iOS updates. Nvidia could bundle them with driver updates. I don't think either would struggle to get people to keep their weights up to date.
    - throwthrowuknow 2mo ago
      I’m sure it would be fine for simple routine tasks but it wouldn’t compete with OpenAI and Anthropic models
      - 2mo ago

philipallstar 2mo ago

This logic seems mad. If people only need SLMs then hyperscalers can also centrally host higher-efficiency models, and still gain efficiencies of scale and convenience over hosting locally.

embedding-shape 2mo ago
You have to remember that articles like these are written for finance people who don't understand the underlying technology, by finance people who don't understand the underlying technology. In this case, the author is a "CFA Institute Enterprising Investor", previously a CIO and basically their entire life been "money, money & money", so hardly surprising they're pulling a lot of assumptions based on what they read. Read the paper the author talks about yourself instead (https://arxiv.org/abs/2511.07885 https://arxiv.org/abs/2511.07885), and also, contrary to what the author says in the article, do not do investments based on single papers made from academic studies, regardless of how much money this guy tells you you can make.
amelius 2mo ago
The logic seems mad to me because SLMs can simply not hold as much information as an LLM. Maybe if you combine an SLM with a database (as a tool) then it could work, but someone should first prove that.
- fph 2mo ago
  But do you really need a model that has the complete Duran Duran discography memorized and preloaded in RAM at all time?
  - amelius 2mo ago
    That's a different question. Probably not. But: 1. Training a large model with lots of information, then stripping the "useless" information from that model to obtain a small model => nobody has shown this. 2. Training a small model, letting it use a database tool so it scores the same as a large model without database => nobody has shown this.

Zigurd 2mo ago

If you are like Google or Apple and you are delivering AI to a mass market unwilling to pay a lot for it, you are absolutely going to drive AI processing to endpoint devices. You are also going to spend what it takes in R&D make a hybrid system that knows when to use local compute or cloud compute. That's going to be the bulk of the workload.

simonebrunozzi 2mo ago

The paper focuses on "intelligence per watt (IPW)", as a way to compare SLMs vs LLMs. What might happen is that a chunk of the market, whatever its size will be, will end up going to SLMs run on iphones or Macbooks, and eat some of the revenues from LLMs, because not everyone needs the most powerful LLM all the time.

eddie_catflap 2mo ago

This is up to October 2025 though, yes? Obviously things are continually moving but Opus 4.5 launched in November and that was a recognised step change in capability. An up to date comparison would be interesting.

Havoc 2mo ago

Complete nonsense. > they provide a better or at least as good an answer as LLMs in 62.5% of the cases. Are we going to scrap hospitals because a vet could do the job 62.5% of the time? The economics also point away from everyone buying a big RAM Mac that sits idle 99% of the time. SLM and own hardware sounds efficient and “free” but it is nothing of the sort when you factor everything in (and forfeit the sharing efficiencies of API) SLMs are great esp for task specific fine tunes but this take isn’t it

physicsguy 2mo ago

One of the big things to think about is whether local LLMs will be things companies want to deploy. If you think of for e.g. some proprietary piece of software that wants to embed an LLM they've fine tuned or trained, they will want to make back some of their research cost right. So they are not going to want to put this on-device even if the hardware is there, unless there's some way of locking it down. I suspect we'll need on-hardware validation/verification and a way of preventing extraction of weights for this move to happen for many use cases.

esseph 2mo ago
> One of the big things to think about is whether local LLMs will be things companies want to deploy. Non-tech enterprise was already doing this years ago. Regulatory reasons, privacy reasons, security, etc. They want on-prem and total ownership of the data. Sometimes air-gapped.
- roryirvine 2mo ago
  Honestly, I've seen a lot less of this than I'd have expected. Lots of discussion, plenty of experiments or toy implementations at the level of individual teams (often driven by one or two enthusiastic individuals), a few proofs of concept for internal services at a more strategic level, but pretty much nothing beyond that. This is in the UK where there's currently a big focus on data sovereignty in general, and I'm genuinely surprised by how little that's spilled over into demand for inference sovereignty (so far). I still expect demand to grow substantially, but I've been saying that for the past couple of years and am beginning to wonder if there'll need to be some sort of trigger event before it happens (eg. the datacentre bubble bursting, or some sort of major scandal).

js8 2mo ago

I believe it is true, and likely there exists a class of even smaller models than what they call "small". You can imagine a reasoning model as a huge set of rules that generate the next statement from previous statements (written in context). In that sense, a reasoning model can be compared to a logical theory - you have certain deduction rules which can generate new judgments. Often, logical theories are structured that the rules are remade into axioms, and the deduction rule is only modus ponens (which corresponds to function application and is a building block of program execution). In the case of an LLM, the set of rules (or axioms) they have in the theory is quite large, but most likely semantically unsound (with respect to their their own representation of truth) - that's why LLM's make mistakes. It would be desirable to break the logical theory represented by LLM into a smaller set of axioms, which would: a) remove rules easily deductible from the smaller core of axioms (for example, LLM doesn't need to remember "Socrates is mortal", as it can derive it from "Socrates is a man" and "all men are mortal") b) remove rules that have low value (facts that aren't used often or have weak validity) which cause ruleset to become unsound I suspect that's what SLM distillation is doing, to some extent. The question is, how far this process can go? I personally believe there is a useful logic for commonsense reasoning that has less than thousand rules (still several orders more than your typical mathematical logic, but orders less than SLMs). These axioms do not contain much facts about the world, but that could be added. So I believe there is a sweet spot (deductive core, encyclopedic shell) which we have not yet found (it's a little bit more formal language than natural language) but is very efficient for general reasoning.

kyleblarson 2mo ago

Given how often the "experts" on CNBC and Bloomberg TV use the term hyperscalers, my approach is to completely disregard anything written or spoken by any person who uses the term.

beepbooptheory 2mo ago
What would be a better term?
- root-parent 2mo ago
  It has to be the Hyperspenders
root-parent 2mo ago
Two weeks ago, CNBC invited one of their "experts" who focus on SpaceX, and he said they have 10 million satellites in orbit. This is the current level of financial journalism available on "specialized" financial channels...

aslkalska 2mo ago

I don't think they are toast, I mean they will be in some trouble because all of them have fallen victim to fomo and started building out with so much debt for capacity that may or may not be needed nor achieve the returns that they want. I think there's a future where "personal software" meaning highly custom apps generated by an agent is a thing that doesn't mean everything will become that, same for local LLMs but all of this is still too far. The main issue is that hyperscalers or big tech in general have become too powerful they can just buy their way in and out of legislation as they please, sorry I mean lobby ... funny how if you rename something it becomes legal or illegal

spinningslate 2mo ago
Exactly. Seems naive at best for an investment consultant to look narrowly at current model capability and not consider the broader market. For example: 1. The hyperscalers are in a positive reinforcement loop. Despite any suggestions to the contrary they keep getting bigger. And can, er, “influence” government policy/officials and anything else needed to keep it that way. 2. The frontier labs and their investors. Another self-fulfilling reinforcement loop. Witness the circular gymnastics among OAI/Anthropic, Microsoft/Amazon and Nvidia 3. Data. No-one believes that Zuckerberg and co are going to say “great, we can just run the models on devices we don’t own and stop the surveillance economy because, y’know, privacy matters and we really care about mental health”. And then there’s data centre locations and “yeah but jobs” even though your power bills are going up, and “why run your own data centre Mrs CTO, let us do it for you and save all that capex and those pesky employees you need to do it”. Don’t get me wrong: I’m rooting for local, open weight/source models. But “hey look they benchmark well” is an unhelpfully narrow basis to forecast the demise of central hyperscaler hosting.
- aslkalska 2mo ago
  oh don't get me started on finance "bros" - they just normalized greed - and that will probably be the doom of us all ... yeah I don't really know what people are doing with these local models - I tried the latest qwen3.8 27b on my decent consumer grade GPU and it struggles to do anything useful when I'm pretty sure Claude would have probably finished the same thing in a couple of hours, like it's not even close.

andai 2mo ago

There's also video models, world models, robotics simulations, the matrix...

andai 2mo ago

Small language model gave satisfactory healthcare output in 100% of cases?

pu_pe 2mo ago

The paper underlying this blog post is fundamentally flawed because of benchmark ceilings. If we define only simple tasks like asking what is the capital of France, all models will converge to 100%, obviously. But as bigger models get more capable we want them to replace more and more complex tasks, in as short time as possible. Then of course there is the economics of it. Do people prefer to spend $5000 upfront to get things done 5x slower, or would they rather pay $20 a month for that?

conorcleary 2mo ago
They'll start labeling scale using panamax-like terminology once they've met and overcome moore's law of this or that, comparing things that hadn't been conceived of yet against an easily memberable/quotable rule.

Garlef 2mo ago

I think one of the watershed developments will be fast models. Imagine current frontier models at 20k tokens/second.

JKCalhoun 2mo ago
I can't read that fast. Maybe this is interesting for an agent?

Jamesbeam 2mo ago

I think it’s a bit more complicated. There is a likely US scenario and a rest of the world scenario. It will be interesting to see if China acts on the overextension of the US Military in the Middle East. Taiwan will be a big play for both and crucial to the hyperscalers. But since the US is dabbling in piracy again and telling people what they can do and not do with their shit, it’s not too far-fetched that everyone that is not a global superpower is at risk of getting bombed to smithereens if they are a danger to US AI supremacy. This is such a crazy timeline, predicting even like a single year ahead feels like looking into a medieval glass ball. But we are humans, I am confident we will find a way to fuck this up royally for everyone. Brace for impact.

1vuio0pswjnm7 2mo ago

"The future could well be specialized edge models that know only ONE thing - and know it well." That design paradigm sounds familiar Everyday I use smalll applications that do only one thing, some written in the 1970's This submission got [flagged]. Later the "[flagged]" label was removed

JKCalhoun 2mo ago
"That design paradigm sounds familiar" In fact sounds like an expert system from decades ago.