For the purpose of generalization, I think that's more treating a symptom than the root cause. The better approach IMO would be finding architectures that heav

For the purpose of generalization, I think that's more treating a symptom than the root cause. The better approach IMO would be finding architectures that heavily penalize the formation of memorizing and interpolating circuits. E.g much stronger weight decay than used today.