5 ms·

The answers here may be of interest: https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https:/

by Jun8 5y ago

The answers here may be of interest: https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https://ai.stackexchange.com/questions/1288/did-minsky-and-p... The unfortunate thing was not the Perceptrons book but the fact that Rosenblatt died prematurely soon after. He was very well-equipped to defend and carry on work on NNs, I think.

savant_penguin 5y ago
The programmers at stackoverflow must be amazing if they fix it with stackoverflow offline
gwern 5y ago
I was reading through the references in https://blogs.umass.edu/comphon/2017/06/15/did-frank-rosenblatt-invent-deep-learning-in-1962/ https://blogs.umass.edu/comphon/2017/06/15/did-frank-rosenbl... linked in another comment, specifically https://www.gwern.net/docs/ai/1993-olazaran.pdf https://www.gwern.net/docs/ai/1993-olazaran.pdf https://www.gwern.net/docs/ai/1996-olazaran.pdf https://www.gwern.net/docs/ai/1996-olazaran.pdf by a Mikel Olazaran who spent a while interviewing what looks to be almost all the surviving connectionists & Minsky etc. Olazaran argues that all the connectionists were perfectly aware of the _Perceptrons_ headline conclusion about single-layer perceptrons being hopelessly linear, which drafts had been circulating for like 4 years beforehand as well, and most regarded it as unimportant (pointing out that humans can't solve the parity of a grid of dots either without painfully counting them out one by one) and having an obvious solution (multiple layers) that they all, Rosenblatt especially, had put a lot of work into trying. The problem was, none of the multi-layer things worked, and people had run out of ideas. So most of the connectionist researchers got sucked away by things that were working at the time (eg the Stanford group was having huge success with adaptive antennas & telephone filters which accidentally come out of their NN work), and funding dried up (for both exogenous political reasons related to military R&D being cut, and just the lack of results compared to alternative research programs like the symbolics approaches which were enjoying their initial flush of success in theorem proving and checkers etc). So when, years later, _Perceptrons_ came out with all of its i's dotted and ts-crossed, it didn't "kill connectionism" because that had already died. What _Perceptrons_ really did was it served as a kind of excuse or Schelling point to make the death 'official' and cement the dominance of the symbolic approaches. Rosenblatt never gave up, but he had already been left high and dry with no more funding and no research community. Olazaran directly asks several of them whether more funding or work would have helped, and it seems everyone agrees that it would've been useless. The computers just weren't there in the '60s. (One notes that it might have worked in the '70s if anyone had paid attention to the invention of backpropagation, pointing out that Rumelhart et al doing the PDP studies on backprop were using the equivalent of PCs for those studies in the late '80s, so if you were patient you could've done them on minicomputers/mainframes in the '70s. But not the '60s.)
neonate 5y ago
https://web.archive.org/web/20201001082122/https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https://web.archive.org/web/20201001082122/https://ai.stacke...

by Jun8 5y ago

savant_penguin 5y ago

The programmers at stackoverflow must be amazing if they fix it with stackoverflow offline

gwern 5y ago

I was reading through the references in https://blogs.umass.edu/comphon/2017/06/15/did-frank-rosenblatt-invent-deep-learning-in-1962/ https://blogs.umass.edu/comphon/2017/06/15/did-frank-rosenbl... linked in another comment, specifically https://www.gwern.net/docs/ai/1993-olazaran.pdf https://www.gwern.net/docs/ai/1993-olazaran.pdf https://www.gwern.net/docs/ai/1996-olazaran.pdf https://www.gwern.net/docs/ai/1996-olazaran.pdf by a Mikel Olazaran who spent a while interviewing what looks to be almost all the surviving connectionists & Minsky etc. Olazaran argues that all the connectionists were perfectly aware of the _Perceptrons_ headline conclusion about single-layer perceptrons being hopelessly linear, which drafts had been circulating for like 4 years beforehand as well, and most regarded it as unimportant (pointing out that humans can't solve the parity of a grid of dots either without painfully counting them out one by one) and having an obvious solution (multiple layers) that they all, Rosenblatt especially, had put a lot of work into trying. The problem was, none of the multi-layer things worked, and people had run out of ideas. So most of the connectionist researchers got sucked away by things that were working at the time (eg the Stanford group was having huge success with adaptive antennas & telephone filters which accidentally come out of their NN work), and funding dried up (for both exogenous political reasons related to military R&D being cut, and just the lack of results compared to alternative research programs like the symbolics approaches which were enjoying their initial flush of success in theorem proving and checkers etc). So when, years later, _Perceptrons_ came out with all of its i's dotted and ts-crossed, it didn't "kill connectionism" because that had already died. What _Perceptrons_ really did was it served as a kind of excuse or Schelling point to make the death 'official' and cement the dominance of the symbolic approaches. Rosenblatt never gave up, but he had already been left high and dry with no more funding and no research community. Olazaran directly asks several of them whether more funding or work would have helped, and it seems everyone agrees that it would've been useless. The computers just weren't there in the '60s. (One notes that it might have worked in the '70s if anyone had paid attention to the invention of backpropagation, pointing out that Rumelhart et al doing the PDP studies on backprop were using the equivalent of PCs for those studies in the late '80s, so if you were patient you could've done them on minicomputers/mainframes in the '70s. But not the '60s.)

neonate 5y ago

https://web.archive.org/web/20201001082122/https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https://web.archive.org/web/20201001082122/https://ai.stacke...