5 ms·
It's not clear that a bunch of cascaded rectified linear functions will every generalize to near 100%. The error floor is at a dangerous level regardless of tra
by forgot-im-old 2y ago
It's not clear that a bunch of cascaded rectified linear functions will every generalize to near 100%. The error floor is at a dangerous level regardless of training. AGI is needed to tackle the final 1%>
- red75prime 2y agoThe universal approximation theorem disagrees. The question is how large the network should be and how much training data it needs. And for now it can only be tested experimentally.
- forgot-im-old 2y agoThe universal approximation theorem does not apply once you include any realistic training algorithms / stochastic gradient descent. There isn't a learnability guarantee.
- red75prime 2y agoThere's no theorem that SGD is insufficient. So, as I said, it's empirical.
- forgot-im-old 2y agoYou said it only depends on network size, I'm saying it more likely is impossible regardless of network size due to fundamental limits in training methods.