4 ms·

That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will wan

That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.

kingleopold 13d ago
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
- leoc 13d ago
  That greater efficiency only benefits the LLM SaaS providers as long as the hardware manufacturers, probably especially the VRAM manufacturers, remain supply constrained, since the high-efficiency users are the ones who can pay top dollar. But the hardware guys' dream is presumably to get parts into millions of laptops which remain on standby for 19 hours a day, not to bargain with SaaS providers who obsessively optimise their memory consumption.
hypfer 13d ago
I'm not sure if this prediction will hold true. We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now. Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
- airspresso 13d ago
  We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.
  - hypfer 13d ago
    Well I mean if I wanted to be extra pedantic, I would argue that we've been in that phase since LLMs were first introduced. Before that, we had 0. After that, we had more than 1. A leap as far as that is hard to recreate. But that wasn't my point. That's just trolling. The actual point is that LLMs aren't gaining new capabilities anymore. They just get more reliable at the ones they already have; turning what was a coin flip to some higher probability. That's (intuitively speaking, not strictly mathematically speaking) kinda the mathematical definition of diminishing returns.
  - bunderbunder 13d ago
    I’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness. But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
    - pixl97 13d ago
      >suspect that most the real gains are actually taking place in the harness. Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.
      - hypfer 13d ago
        That is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might. LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker. But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
- 48488448 13d ago
  they really dont want to hear this bro lol
  - hypfer 13d ago
    I can see that by those reddit-style vote swings, but who are "they", exactly? Who is so emotionally invested into random comment sections being purely positive about their pet.. uuuuuuuh.. tech? Very weird.
  - IshKebab 12d ago
    On the contrary I would love it if AI stagnates. I don't want to be out of a job. But I also don't believe things just because I want them to be true.
- gehsty 13d ago
  It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
  - hypfer 13d ago
    Hmm I'm not 100% sure about that, given that edge is very viable, and the geopolitical climate has changed quite significantly. I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.
bunderbunder 13d ago
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either. It may turn out that demand for SOTA frontier models isn’t so limitless after all.
- pixl97 13d ago
  Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.
  - bunderbunder 13d ago
    Except of course it relates. All else being equal, we will prefer $X COGS over $2X COGS because that helps us with both profit margins and price competition.
    - pixl97 13d ago
      It relates in the sense there's a minimum cost of production without losses, not the actual price people are willing to pay.
      - bunderbunder 13d ago
        Framing it in terms of the price people might be willing to pay for a single product in isolation frames the point I was making, which was about price competition, right out of the picture. Maybe I'd be willing to pay $10 for product A if I had other options. But if there's a product B for $3 that's not quite as nice but still ticks all my boxes, then product instantly becomes a lot less attractive.
  - trombuance 13d ago
    >At a price of 0 you could find infinite usage Infinite demand isn't a thing. Even if they'd offer free compute forever (not likely possible) I and many others would still use local models that we have full control over and that do not harvest our personal data.
blurbleblurble 13d ago
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
- IshKebab 13d ago
  > It's going to happen very soon Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.
  - dwedge 13d ago
    You asserted that it was never going to happen first
    - geysersam 13d ago
      But he gave a reason for that. "Before that happens the frontier will move". Why do you think it will happen anyway? Do you think the frontier will not move fast enough that local models are unable to catch up, or do you think people will prefer local models at a point. Or something else?
  - blurbleblurble 13d ago
    Because everything is converging on a backlog of huge efficiency gains established in research, waiting to be combined. Looped transformers, a whole host of diffusion techniques and new quantization techniques, maturation of ternary distillation and new ways to separate logic from stuff that can be looked up. It would surprise me if most frontier models were actually even that big at that point in terms of active params. I highly doubt it.
- mrlonglong 13d ago
  It is for that reason they are being archived and torrented as a very large middle finger.
- deleted 13d ago
dabinat 13d ago
You don’t need frontier-level performance for every task. That’s why companies hire both junior and senior developers. I suspect a decent percentage of people using Fable would probably be fine with Opus. Also, the frontier can’t keep advancing at this rate forever. Eventually the low-hanging fruit will all be gone and advances will slow down.

That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.

kingleopold 13d ago

This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price

leoc 13d ago
That greater efficiency only benefits the LLM SaaS providers as long as the hardware manufacturers, probably especially the VRAM manufacturers, remain supply constrained, since the high-efficiency users are the ones who can pay top dollar. But the hardware guys' dream is presumably to get parts into millions of laptops which remain on standby for 19 hours a day, not to bargain with SaaS providers who obsessively optimise their memory consumption.

hypfer 13d ago

I'm not sure if this prediction will hold true. We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now. Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.

airspresso 13d ago
We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.
- hypfer 13d ago
  Well I mean if I wanted to be extra pedantic, I would argue that we've been in that phase since LLMs were first introduced. Before that, we had 0. After that, we had more than 1. A leap as far as that is hard to recreate. But that wasn't my point. That's just trolling. The actual point is that LLMs aren't gaining new capabilities anymore. They just get more reliable at the ones they already have; turning what was a coin flip to some higher probability. That's (intuitively speaking, not strictly mathematically speaking) kinda the mathematical definition of diminishing returns.
- bunderbunder 13d ago
  I’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness. But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
  - pixl97 13d ago
    >suspect that most the real gains are actually taking place in the harness. Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.
    - hypfer 13d ago
      That is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might. LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker. But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
48488448 13d ago
they really dont want to hear this bro lol
- hypfer 13d ago
  I can see that by those reddit-style vote swings, but who are "they", exactly? Who is so emotionally invested into random comment sections being purely positive about their pet.. uuuuuuuh.. tech? Very weird.
- IshKebab 12d ago
  On the contrary I would love it if AI stagnates. I don't want to be out of a job. But I also don't believe things just because I want them to be true.
gehsty 13d ago
It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
- hypfer 13d ago
  Hmm I'm not 100% sure about that, given that edge is very viable, and the geopolitical climate has changed quite significantly. I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.

bunderbunder 13d ago

I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either. It may turn out that demand for SOTA frontier models isn’t so limitless after all.

pixl97 13d ago
Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.
- bunderbunder 13d ago
  Except of course it relates. All else being equal, we will prefer $X COGS over $2X COGS because that helps us with both profit margins and price competition.
  - pixl97 13d ago
    It relates in the sense there's a minimum cost of production without losses, not the actual price people are willing to pay.
    - bunderbunder 13d ago
      Framing it in terms of the price people might be willing to pay for a single product in isolation frames the point I was making, which was about price competition, right out of the picture. Maybe I'd be willing to pay $10 for product A if I had other options. But if there's a product B for $3 that's not quite as nice but still ticks all my boxes, then product instantly becomes a lot less attractive.
- trombuance 13d ago
  >At a price of 0 you could find infinite usage Infinite demand isn't a thing. Even if they'd offer free compute forever (not likely possible) I and many others would still use local models that we have full control over and that do not harvest our personal data.

blurbleblurble 13d ago

It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.

IshKebab 13d ago
> It's going to happen very soon Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.
- dwedge 13d ago
  You asserted that it was never going to happen first
  - geysersam 13d ago
    But he gave a reason for that. "Before that happens the frontier will move". Why do you think it will happen anyway? Do you think the frontier will not move fast enough that local models are unable to catch up, or do you think people will prefer local models at a point. Or something else?
- blurbleblurble 13d ago
  Because everything is converging on a backlog of huge efficiency gains established in research, waiting to be combined. Looped transformers, a whole host of diffusion techniques and new quantization techniques, maturation of ternary distillation and new ways to separate logic from stuff that can be looked up. It would surprise me if most frontier models were actually even that big at that point in terms of active params. I highly doubt it.
mrlonglong 13d ago
It is for that reason they are being archived and torrented as a very large middle finger.
deleted 13d ago

dabinat 13d ago

You don’t need frontier-level performance for every task. That’s why companies hire both junior and senior developers. I suspect a decent percentage of people using Fable would probably be fine with Opus. Also, the frontier can’t keep advancing at this rate forever. Eventually the low-hanging fruit will all be gone and advances will slow down.