3 ms·
There are many indications that model progress is slowing down, so that is not entirely accurate.
by civvv 6mo ago
There are many indications that model progress is slowing down, so that is not entirely accurate.
- StrauXX 6mo agoWhich indications are that?
- overfeed 6mo agoInvestment dollars.
- dzhiurgis 6mo agoSource for that claim?
- deleted 6mo ago[deleted]
- lionkor 6mo agoNobody is releasing NEW models
- taneq 6mo agoThe standard networking connection has been called “Ethernet” for more than thirty years, so networking has stagnated, right?
- SlinkyOnStairs 6mo agoIf higher bandwidth networking consisted primarily running more and more ethernet lines in parallel, you would most certainly agree that "networking has stagnated". "Reasoning" and now "Agentic" AI systems are not some fundamental improvement on LLMs, they're just running roughly the same prior-gen LLMS, multiple times. Hence the conclusion that LLM improvement has slowed down, if not stagnated entirely, and that we should not expect the improvements of switching to these "reasoning" systems to keep happening.
- p1esk 6mo agoFrom TFA: “ChatGPT came up with an idea which is original and clever. It is the sort of idea I would be very proud to come up with after a week or two of pondering, and it took ChatGPT less than an hour to find and prove”
- SlinkyOnStairs 6mo agoYou misunderstand. I'm not saying that Reasoning/Agentic systems aren't better. I'm saying they're not an advancement in the tech in the way GPT 1 through 3 were. They're a different kind of improvement. And as such the rate improvement cannot just be extrapolated into the future.
- p1esk 6mo agoGPT1 through GPT3 advancement were exactly like using more Ethernet cables in parallel. All interesting conceptual breakthroughs came after GPT3: RL and reasoning being the main ones.
- kstenerud 6mo agoWhat constitutes a NEW model for the purposes of calculating progress?
- aspenmartin 6mo ago…not only is this not true but it also doesn’t matter. Why would this indicate performance saturating?
- GardenLetter27 6mo agoWhat? DeepSeekV3 just came out and is incredible for the price. Mythos is also half-released.
- nozzlegear 6mo agoUntil you or I can actually use Mythos in Claude without an nda or other strings attached, Mythos is not released and is just an effective marketing tool for Anthropic.
- pixl97 6mo agoAt least to me this is a pretty sour grapes take. There are all kinds of released products that are expensive or need an NDA. You're just too poor to afford it. But make no mistakes there are governments using this in mass and likely against you.
- nxobject 6mo agoI think that’s worthy of at least sour grapes, too.
- nicoburns 6mo agoThe cost factors on the new models compared to the old models.
- bdelmas 6mo agoYou are mixing cost and progress. It’s not because it’s more and more expensive that progress is slowing down by itself.
- nicoburns 6mo agoThey are intrinsically linked beyond a certain point. If we're making progress but costs are spiraling exponentially then it stands to reason that we will soon reach a point where we can no longer afford the increasing costs and thus progress will slow. (barring some breakthrough that reduces costs, which of course may happen, but for which recent model improvements are not strong evidence of)
- jeremyjh 6mo agoQwen3.6 9B is as good as GPT-4o and runs on my M2 MacBook Air. Models are getting stronger and less costly at the same time, but these are somewhat separate branches of research. Frontier labs are spending more because they are still getting marginal returns and there is more capacity to spend than there was a year ago.
- gertop 6mo agoQwen 3.6 9B doesn't exist. If you meant 3.5 9B and you truly believe it's as good as 4o then I can only assume you have a very basic use case.
- jeremyjh 6mo agoYou are right, I was mistaken about the version. I evaluated it in general chat assistant prompts plucked from my history across a range of topics but did not use it for coding - there was never a time when I thought 4o was “good enough” for agentic coding.
- aspenmartin 6mo agoPlease be specific because outside of anecdotal blog posts by people who don’t know what they’re talking about it’s not true. Look at scaling laws, composite benchmarks from the epoch capability index, nothing at all suggests “model progress is slowing down”
- CuriouslyC 6mo agoModel progress at spitting out unhallucinated facts is slowing down hard. Model progress at solving hard math challenges/programming tasks doesn't seem to be slowing down that I can tell.