4 ms·

It's not clear to me that each new generation of models is going to be "that" much better vs cost. Anecdotally moving from model to model I'm not seeing huge c

• ljlolel 1y ago
  The scaling laws already predict diminishing in returns
• renegade-otter 1y ago
  We do seem to be hitting the top of the curve of diminishing returns. Forget AGI - they need a performance breakthrough in order to stop shoveling money into this cash furnace.
  • jayde2767 1y ago
    "cash furnace", so aptly put.
    • gizajob 1y ago
      The economics will work out when the district heating is run off the local AI/cash furnace.
    • nielsbot 1y ago
      And don't forget the furnace furnace: gas/coal to power all this.
  • general1465 1y ago
    Yep we do. There is a 1 year old video on YouTube, which describes this limitation https://www.youtube.com/watch?v=5eqRuVp65eY https://www.youtube.com/watch?v=5eqRuVp65eY Called efficient compute frontier
  • fredoliveira 1y ago
    I think that the performance unlock from ramping up RL (RLVR specifically) is not fully priced into the current generation yet. Could be wrong, and people closer to the metal will know better, but people I talk to still feel optimistic about the next couple of years.
  • duxup 1y ago
    >cash furnace They don't even burn it on on AI all the time either: https://openai.com/sam-and-jony/ https://openai.com/sam-and-jony/
    • dmbche 1y ago
      "May 21, 2025 This is an extraordinary moment. Computers are now seeing, thinking and understanding. Despite this unprecedented capability, our experience remains shaped by traditional products and interfaces." I don't even want to learn about them every line is so exhausting
      • duxup 1y ago
        Agreed, that whole page is brutal to read.
    • serf 1y ago
      I was expecting a wedding or birth announcement from that picture framing and title. "We would like to introduce you to the spawn of Johnny Ive and Sam Altman, we're naming him Damien Thorn."
  • mikestorrent 1y ago
    Inference performance per watt is continuing to improve, so even if we hit the peak of what LLM technology can scale to, we'll see tokens per second, per dollar, and per watt continue to improve for a long time yet. I don't think we're hitting peak of what LLMs can do, at all, yet. Raw performance for one-shot responses, maybe; but there's a ton of room to improve "frameworks of thought", which are what agents and other LLM based workflows are best conceptualized as. The real question in my mind is whether we will continue to see really good open-source model releases for people to run on their own hardware, or if the companies will become increasingly proprietary as their revenue becomes more clearly tied up in selling inference as a service vs. raising massive amounts of money to pursue AGI.
    • ethbr1 1y ago
      My guess would be that it parallels other backend software revolutions. Initially, first party proprietary solutions are in front. Then, as the second-party ecosystem matures, they build on highest-performance proprietary solutions. Then, as second parties monetize, they begin switching to OSS/commodity solutions to lower COGS. And with wider use, these begin to outcompete proprietary solutions on ergonomics and stability (even if not absolute performance). While Anthropic and OpenAi are incinerating money, why not build on their platforms? As soon as they stop, scales tilt towards an apache/nginx type commoditized backend.
  • reissbaker 1y ago
    According to Dario, each model line has generally been profitable: i.e. $200MM to train a model that makes $1B in profit over its lifetime. But, since each model has been more and more expensive to train, they keep needing to raise more money to train the next generation of model, and the company balance sheet looks negative: i.e. they spent more this year than last (since the training cost for model N+1 is higher), and the model this year made less money this year than they spent (even if the model generation itself was profitable, model N isn't profitable enough to train model N+1 without raising — and spending — more money). That's still a pretty good deal for an investor: if I give you $15B, you will probably make a lot more than $15B with it. But it does raise questions about when it will simply become infeasible to train the subsequent model generation due to the costs going up so much (even if, in all likelihood, that model would eventually turn a profit).
    • viscanti 1y ago
      Well how much of it is correlation vs causation. Does the next generation of model unlock another 10x usage? Or was Claude 3 "good enough" that it got traction from early adopters and Claude 4 is "good enough" that it's getting a lot of mid/late adopters using it for this generation? Presumably competitors get better and at cheaper prices (Anthropic charges a premium per token currently) as well.
    • dom96 1y ago
      > if I give you $15B, you will probably make a lot more than $15B with it "probably" is the key word here, this feels like a ponzi scheme to me. What happens when the next model isn't a big enough jump over the last one to repay the investment? It seems like this already happened with GPT-5. They've hit a wall, so how can they be confident enough to invest ever more money into this?
      • bcrosby95 1y ago
        I think you're really bending over backwards to make this company seem non viable. If model training has truly turned out to be profitable at the end of each cycle, then this company is going to make money hand over fist, and investing money to out compete the competition is the right thing to do. Most mega corps started out wildly unprofitable due to investing into the core business... until they aren't. It's almost as if people forget the days of Facebook being seen as continually unprofitable. This is how basically all huge tech companies you know today started.
• yieldcrv 1y ago
  Locally run video models that are just as good as today’s closed models are going to be the watershed moment The companies doing foundational video models have stakeholders that don’t want to be associated with what people really want to generate But they are pushing the space forward and the uncensored and unrestricted video model is coming
  • giancarlostoro 1y ago
    Nobody wants to make a commercial NSFW model that then suffers a jailbreak... for what is the most illegal NSFW content.
    • yieldcrv 1y ago
      Thats the thing, what’s “illegal” will challenge our whole society when it comes do dynamically generated real interactive avatars that are new humans When it comes to sexually explicit content in general with adults, all of our laws rely on the human actor existing FOSTA and SESTA is related to user generated content of humans, for example. They rely on making sure an actual human isnt being exploited and burdening everyone with that enforcement. When everyone can just say “thats AI” nobody’s going to care and platforms will be willing to take that risk of it being true again - or a new hit platform will. That kind of content currently Doesnt exist in large quantities yet, until a video model ungimped can generate it. Concerns about trafficking only rely on actual humans not entirely new avatars regarding children there are more restrictions that may already cover this, there is a large market for just adult looking characters though and worries about underage can be tackled independently. or be found entirely futile. not my problem, focus on what you can control. this is whats coming though. people already dont mind parasocial relationships with generative AI and already pay for that, just add nudity
    • simianwords 1y ago
      Why is this illegal btw? I mean whats stopping an AI company from releasing a proper NSFW model? I hope it doesn't happen but I want to know what prevents them from doing it now.
• ACCount37 1y ago
  The raw model scale is not increasing by much lately. AI companies are constrained by what fits in this generation of hardware, and waiting for the next generation to become available. Models that are much larger than the current frontier are still too expensive to train, and far too expensive to serve them en masse. In the meanwhile, "better data", "better training methods" and "more training compute" are the main ways you can squeeze out more performance juice without increasing the scale. And there are obvious gains to be had there.
  • xnx 1y ago
    > AI companies are constrained by what fits in this generation of hardware, and waiting for the next generation to become available. Does this apply to Google that is using custom built TPUs while everyone else uses stock Nvidia?
    • ACCount37 1y ago
      By all accounts, what's in Google's racks right now (TPU v5e, v6e) is vaguely H100-adjacent, in both raw performance and supported model size. If Google wants anything better than that? They, too, have to wait for the new hardware to arrive. Chips have a lead time - they may be your own designs, but you can't just wish them into existence.
      • xxpor 1y ago
        Aren't chips + memory constrained by process + reticle size? And therefore, how much HBM you can stuff around the compute chip? I'd expect everyone to more or less support the same model size at the same time because of this, without a very fundamentally different architecture.
  • robwwilliams 1y ago
    The jump to 1 million token length context for Sonnet 4 plus access to internet has been a game-changer for me. And somebody should remind Anthropic leadership to at least mirror Wikipedia; better yet support Wikipedia actively. All of the big AI players have profited from Wikipedia, but have they given anything back, or are they just parasites on FOSS and free data?
• gmadsen 1y ago
  Its not clear to me that it needs to. If at the margins it can still provide an advantage in the market or national defense, then the spice must flow
  • duxup 1y ago
    I suspect it needs to if it is going to cover the costs of training.
• derefr 1y ago
  > Anecdotally moving from model to model I'm not seeing huge changes in many use cases. Probably because you're doing things that are hitting mostly the "well-established" behaviors of these models — the ones that have been stable for at least a full model-generation now, that the AI bigcorps are currently happy keeping stable (since they achieved 100% on some previous benchmark for those behaviors, and changing them now would be a regression per those benchmarks.) Meanwhile, the AI bigcorps are focusing on extending these models' capabilities at the edge/frontier, to get them to do things they can't currently do. (Mostly this is inside-baseball stuff to "make the model better as a tool for enhancing the model": ever-better domain-specific analysis capabilities, to "logic out" whether training data belongs in the training corpus for some fine-tune; and domain-specific synthesis capabilities, to procedurally generate unbounded amounts of useful fine-tuning corpus for specific tasks, ala AlphaZero playing unbounded amounts of Go games against itself to learn on.) This means that the models are getting constantly bigger. And this is unsustainable. So, obviously, the goal here is to go through this as a transitionary bootstrap phase, to reach some goal that allows the size of the models to be reduced. IMHO these models will mostly stay stable-looking for their established consumer-facing use-cases, while slowly expanding TAM "in the background" into new domain-specific use-cases (e.g. constructing novel math proofs in iterative cooperation with a prover) — until eventually, the sum of those added domain-specific capabilities will turn out to have all along doubled as a toolkit these companies were slowly building to "use models to analyze models" — allowing the AI bigcorps to apply models to the task of optimizing models down to something that run with positive-margin OpEx on whatever hardware that would be available at that time 5+ years down the line. And then we'll see them turn to genuinely improving the model behavior for consumer use-cases again; because only at that point will they genuinely be making money by scaling consumer usage — rather than treating consumer usage purely as a marketing loss-leader paid for by the professional usage + ongoing capital investment that that consumer usage inspires.
  • kdmtctl 1y ago
    You have just described a singularity point for this line of business. Which could happen. Or not.
    • 1y ago
• darepublic 1y ago
  I hope you're right.
• wslh 1y ago
  > Anecdotally moving from model to model I'm not seeing huge changes in many use cases. I can just pick an older model and often I can't tell the difference... Model specialization. For example a model with legal knowledge based on [private] sources not used until now.
• dvfjsdhgfv 1y ago
  > I can just pick an older model and often I can't tell the difference... Or, as in the case of a leading North American LLM provider, I would love to be able to choose an older model but it chooses it for me instead.