3 ms·
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function Incor
by CamperBob2 3y ago
This keeps popping up, but a sufficiently large neural network can approximate any continuous function. They can't approximate any discontinuous function
Incorrect. That's what nonlinear activation functions are for.
Researchers wasted many potentially-fruitful years because Minsky and other luminaries -- people who occupied the same position of authority that LeCun occupies today -- made a huge deal about how the original perceptron and subsequent multilayer variants "couldn't learn XOR." That was almost literally how they put it. At the first sign of difficulty they abandoned the approach that was, and remains, the most promising. We have to be careful to learn from that mistake.