2 ms·
What about the first algorithm for back-propagating neural networks? I can't remember who did that but I am sure someone on HN will remember. When Minsky's "P
by georgeecollins 8y ago
What about the first algorithm for back-propagating neural networks? I can't remember who did that but I am sure someone on HN will remember. When Minsky's "Perceptrons" was written there were all these problems with neural networks that back-propagation solved. That seems like kind of a big deal. It also seems like a lot deep learning is built off of or related to that work.*
* I am not that knowledgeable about deep learning so please correct me if I am misinformed.
- stabbles 8y agoOut of genuine curiosity, what exactly is novel about this? Isn't back-propagation just the chain rule (i.e. as old as calculus). And I figure 'gradient descent' was discovered not long after?
- georgeecollins 8y agoI am not an expert, but Marvin Minsky wrote a whole book about the kinds of problems that Neural Networks can't solve, that didn't take into account back-propagation. No one refuted it, as far as I know for a long time. This may all seem obvious now but I don't think anyone actually did it until the mid-eighties.