3 ms·
> can conceivably fit a curve with a single, huge layer I think you need a hidden layer. I’ve never seen a universal approximation theorem for a single layer
by mr_toad 2y ago
> can conceivably fit a curve with a single, huge layer
I think you need a hidden layer. I’ve never seen a universal approximation theorem for a single layer network.
- dongecko 2y agoI second that thought. There is a pretty well cited paper from the late eighties called "Multilayer Feedforward Networks are Universal Approximators". It shows that a feedforward network with a single hidden layer containing a finite number of neurons can approximate any continuous function. For non continous function additional layers are needed.
- ted_dunning 2y agoMinsky and Papert showed that single layer perceptrons suffer from exponentially bad scaling to reach a certain accuracy for certain problems. Multi-layer substantially changes the scaling.