4 ms·
A lot of the arguments made in this article are based on debatably true assumptions. For example, I don't think most advances in AI in recent years has been dri
by kypro 4y ago
A lot of the arguments made in this article are based on debatably true assumptions. For example, I don't think most advances in AI in recent years has been driven by increases in compute and data, but more so by refinements in how we train models.
Active learning would be useful at creating a superhuman AGI, but I don't see it as a requirement. As soon as AIs can iterate on future models and produce increasingly useful training data for future models, then models would likely continue to grow more capable at an exponential rate.
But I'm not even convinced active learning is that difficult once you have fairly capable AI programmers...
My AI professor once said, "all problems are search problems". And this suck with me because the world looks different when you start seeing all problems as just iterations of different optimisation problems. So long as you have a good heuristic and enough compute, eventually a suitable solution for any problem will be found. This is what makes generalised learning algorithms so powerful.
The author states that, "active learning with neural networks are uniformly terrible", but this just reeks to me of a search problem in need of optimisation. So as long as we can tell GPT5, "I want you to iteratively write code which improve on existing state-of-the-art active learning algorithms", then eventually you'll get there... GPT5 like humans might not get very far in the beginning, but assuming it can iterate faster than humans and future iterations faster still, then breakthroughs will eventually be made and progress should accelerate exponentially.
Of course, this also assumes that humans don't get there first. And honestly given the amount of investment in AI and rate of progress in AI right now I just wouldn't bet against a significant advancement in active learning coming soon.