RE: 1-layer neural nets, yes, that's the point. Deep learning has been a series of *engineering* successes stacking over each other rather than theory being ap

RE: 1-layer neural nets, yes, that's the point. Deep learning has been a series of *engineering* successes stacking over each other rather than theory being applied rigorously. It's hard to scale training on the "dumb" approximators like a kitchen sink regression, and balancing overfitting becomes a nightmare.