A lot of people seem to take the rapid improvement of LLMs from GPT-2 through GPT-4 and their brethren, and extrapolate that trendline to infinity. But that's

A lot of people seem to take the rapid improvement of LLMs from GPT-2 through GPT-4 and their brethren, and extrapolate that trendline to infinity. But that's not logically sound. The advances that have allowed this aren't arbitrarily scalable. Sure, we may see some more advances in AI tech that take us a few more jumps forward—but that doesn't imply that we will keep advancing at this pace until we hit AGI/superintelligence/the singularity/whatever. I've seen several people compare this logic to what we were seeing in the discussions about self-driving technology several years ago: some very impressive advancements had happened, and were continuing to happen, and so people extrapolated from there to assume that full self-driving capability would be coming to the market by...well, about now, actually. (I admit, I somewhat bought the hype at that time. It is possible this makes me feel more cautious now; YMMV.) I find this comparison to be persuasive, as it touches on some very similar improvements in technology. I believe that we will see ML advancements hit a similar wall fairly soon.