Many good points in the article, but I'd caveat that if you judge performance only by ELO score, you are not applying the best criteria. SWEBench performance m

Many good points in the article, but I'd caveat that if you judge performance only by ELO score, you are not applying the best criteria. SWEBench performance moved from 25% to 70% solved since beginning of 2025, and even with the narrowest possible lens from 65% to 70% since May. ARC-AGI2 keeps rapidly climing. We have experimental models able to (maybe) hold their ground at IMO gold. As well as performing at IPhO gold level. And that leaves out the point that LMArena is a popularity contest. Not "was this correct", but "which answer did you like better". The thing that brought us glazing. A leveling ELO (or a very slowly climbing one) is kind of expected, and is really saying nothing about progress in the field. Still doesn't mean "THE MACHINE GODS ARE COMING", but I would expect to see continued if slowing improvement. (I mean, how much better can you get at math if you already can be a useful assistant to Terence Tao and win IMO gold? And wouldn't we expect that progress to slow?) But more than "how hard of a problem can you solve", I expect we'll see a shift to instead looking at missing capabilities. E.g. memory - currently, you can tell Claude 500 times to use uv, not pip, and it will cheerfully not know that again the 501st time. That's much more important than "oh, it now solves slightly harder problems most of us don't have". And if you look at arxiv papers, a lot are peeking in that direction. I'd also expect work on efficiency. "can we not make it cost about the amount of India's budget to move the industry forward, every year" is kind of a nice idea. No, I'm not making the number up, or taking OAIs fantasy numbers - 2025 AI industry capex is expected to be $375B. We'll likely need that efficiency if we want to get significantly better at difficulty level or task length, too.