The same could be said of transformers, that only started to perform when scaled up to an absolutely ridiculous degree. I would argue most researchers are of th

The same could be said of transformers, that only started to perform when scaled up to an absolutely ridiculous degree. I would argue most researchers are of the opinion that any learning system, scaled up enough, would work. I think the limits of machine learning are related to the fact that all ML "knowledge" is secondhand, except talking to humans and already to a much smaller extent programming. Getting AIs to interact with, say, cars, during training is the way forward.