4 ms·
I'm not sure if this prediction will hold true. We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still
by hypfer 6d ago
I'm not sure if this prediction will hold true.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
- airspresso 6d agoWe are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.
- hypfer 6d agoWell I mean if I wanted to be extra pedantic, I would argue that we've been in that phase since LLMs were first introduced. Before that, we had 0. After that, we had more than 1. A leap as far as that is hard to recreate. But that wasn't my point. That's just trolling. The actual point is that LLMs aren't gaining new capabilities anymore. They just get more reliable at the ones they already have; turning what was a coin flip to some higher probability. That's (intuitively speaking, not strictly mathematically speaking) kinda the mathematical definition of diminishing returns.
- bunderbunder 6d agoI’ll grant that for specialized applications like coding agents and mathematics, but even there I suspect that most the real gains are actually taking place in the harness. But I suspect returns may have already diminished into negative territory for at least some other use cases. One of my least favorite job responsibilities in this brave new era is figuring out how to avoid performance and behavior regressions when an older model were using for some application reaches end of life. It’s getting uncommon for me to look at our benchmark results and say, “Oh, good, it does better on one of the newer models!”
- pixl97 6d ago>suspect that most the real gains are actually taking place in the harness. Part of the reason harnesses work well is you can run a lot of agents in parallel. That doesn't slow down demand.
- hypfer 6d agoThat is true, but the eventual realization that more machines doing more coin flips in parallel does not mean "more work gets done" might. LLMs are amazing tech, but they're terrible without oversight. More agents faster just makes reality collapse on them quicker. But yeah, you're right, temporarily, this will still push demand. But the topic was about "diminishing returns" as in "tech getting better". Not as in "customer spending".
- pixl97 6d agoIt's kind of weird because more machines working together does mean more work gets done. Coin flips and weighted coin flips are totally different things. Any biases weights towards reality push you closer to reality when you use them. New models keep being able to use more and more agents on longer time frames. Your hypothesis doesn't look like what we're measuring.
- SideQuark 5d agoGoogle scholar has a flood of papers showing LLM diminishing returns on pretty much every facet https://scholar.google.com/scholar?hl=en&as_sdt=0%2C23&q=llm+diminishing+returns+&btnG= https://scholar.google.com/scholar?hl=en&as_sdt=0%2C23&q=llm...
- T-A 5d agoThe first title I see there is "The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs": https://proceedings.iclr.cc/paper_files/paper/2026/hash/3b4e1336f775c3dba16ebbb8d2afd258-Abstract-Conference.html https://proceedings.iclr.cc/paper_files/paper/2026/hash/3b4e...
- 48488448 6d agothey really dont want to hear this bro lol
- hypfer 6d agoI can see that by those reddit-style vote swings, but who are "they", exactly? Who is so emotionally invested into random comment sections being purely positive about their pet.. uuuuuuuh.. tech? Very weird.
- IshKebab 5d agoOn the contrary I would love it if AI stagnates. I don't want to be out of a job. But I also don't believe things just because I want them to be true.
- gehsty 6d agoIt’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
- hypfer 6d agoHmm I'm not 100% sure about that, given that edge is very viable, and the geopolitical climate has changed quite significantly. I agree that datacenters are not going to go away, but I have doubts that the buildup that has happened is really going to pay off for most operators.