Of course, I agree with you that the parameter space is ridiculously large. But sadly, that's what people do in practice. And with 20mio, their example is still

Of course, I agree with you that the parameter space is ridiculously large. But sadly, that's what people do in practice. And with 20mio, their example is still small in comparison to GPT-3 with 175 billion parameters. I disagree with you on the restrictive part. Those ML models that are inspired by biology tend to be restrictive, the same way that the development of mammal brains is assumed to be restricted by genetically determined structure. Pretty much all SOTA optical flow algorithms are restricted in what they can learn. And those restrictions are what makes convergence and unsupervised learning possible, because the problem by itself is very ill posed.