4 ms·
It's funny because the author doesn't realize that this sentence at the beginning undermines his entire argument: > Or they see two consecutive model releases
by littlestymaar 1y ago
It's funny because the author doesn't realize that this sentence at the beginning undermines his entire argument:
> Or they see two consecutive model releases and don’t notice much difference in their conversations, and they conclude that AI is plateauing and scaling is over.
The reason why we now fail to notice the difference between consecutive models now is because the progress isn't in fact exponential. Humans tend to have a logarithmic perception, which means we only appreciate progress when it is exponential (for instance you'd be very happy to get a $500 raise if you are living the minimum wage, but you wouldn't even call that “a raise” when on a SV engineers salary).
AI models have been improving a ton for the past three years, in many direction, but the rate of progress is definitely not exponential. It's not emergent either, as the focus is now being specifically directed at solving specific problems (both riddles and real world problems) thanks to trillions of token of high quality synthetic data.
On topics that aren't explicitly being worked on, progress have been minimal or even negative (for instance many people still use the 1 year old Mistral Nemo for creative writing because the more recent ones all have been STEMmaxxed)