5 ms·
The LLMs rate of improvement has really slowed down. This looks like a minor improvement in terms of accuracy and big gains from efficiency.
by ddxv 11mo ago
The LLMs rate of improvement has really slowed down. This looks like a minor improvement in terms of accuracy and big gains from efficiency.
- energy123 11mo ago14 months ago we had GPT-4 and now we have models that can get a gold medal at the IMO. But sure, if you curve fit to the last 3 months you could say things are slowing down, but that's hyper fixating on a very small amount of information.
- ddxv 11mo agoYes, that is what I'm saying, that 14 months ago the rate of change was noticeably faster. Lately the new models are much less groundbreaking and increasing in the volume of output and decreasing in cost.
- energy123 11mo agoThe private model that got gold at IMO was 4 months ago. 14 months ago we had o1-preview, we didn't have that gold medal winning approach yet. You could only say that things have slowed down since 4 months ago, but in my view that's reading the tea leaves too much. It's just not enough time and too little visibility into the private research.
- riku_iki 11mo agoit could be results of corps focusing resources on IMO in PR wars, and results is not as generalizable outside this niche.