3 ms·
OP failing to understand S-curves again... I think the first comment on the article put it best: With COVID, researchers could be certain that exponential grow
by xg15 1y ago
OP failing to understand S-curves again...
I think the first comment on the article put it best: With COVID, researchers could be certain that exponential growth was taking place because they knew the underlying mechanisms of the growth. The virus was self-replicating, so the more people were already infected, the faster would new infections happen.
(Even this dynamic would only go on for a certain time and eventual slow down, forming an S-curve, when the virus could not find any more vulnerable persons to continue the rate of spread. The critical question was of course if this would happen because everyone was vaccinated or isolated enough to prevent infection - or because everyone was already infected or dead)
With AI, there is no such underlying mechanism. There is the dream of the "self-improving AI" where either humans can make use of the current-generation AI to develop the next-generation AI in a fraction of the time - or where the AI simply creates the next generation on its own.
If this dream were reality, it could be genuine exponential growth, but from all I know, it isn't. Coding agents speed up a number of bespoke programming tasks, but they do not exponentially speed up development of new AI models. Yes, we can now quickly generate large corpora of synthetic training data and use them for distillation. We couldn't do that before - but a large part of the training data discussion is about the observation that synthetic data can not replace real data, so data collection remains a bottleneck.
There is one point where a feedback loop does happen, and this is with the hype curve: Initial models produced extremely impressive results compared to everything we had before - there caused an enormous hype and unlocked investments that allowed more resources for the developed of the next model - which then delivered even better results. But it's obvious that this kind of feedback loop will eventually end when no more additional capital is available and diminishing returns set in.
Then we will once again be in the upper part of the S-curve.