4 ms·
Over-parameterization for regularization is really old. The pseudoinverse min-norm solution for under-determined linear systems even has that flavor.
by pishpash 5y ago
Over-parameterization for regularization is really old. The pseudoinverse min-norm solution for under-determined linear systems even has that flavor.
- tomrod 5y agoSure, but that's identification approaches in econometrics and matrix analysis contexts. Using that for neural networks is new-ish in the zeitgeist, which did not exist in the 1990s as it does today.