"Generalization" is always a data problem. If you trained it on one function class, of course that's all it learned to do. That's all it ever saw! If you want

"Generalization" is always a data problem. If you trained it on one function class, of course that's all it learned to do. That's all it ever saw! If you want to learn arbitrary function classes to some degree, the solution is simple. Train it on many different function classes. Untrained models are as blank slate as you could possibly imagine. They're not even comparable to new born humans with millions of years of evolution baked in. The data you feed them is their world. Their only world.