4 ms·

It's just such a shame that Minksy came out so harshly against perceptrons. He really was an inspiring figure, and at the time really did seem to have good reas

• varjag 10y ago
  Mind you the "successes of NNs" didn't start to show up until 2010 or so, despite active research in multi-layer ANNs going from early 1980s. Just the past decade ANN classifier performance wasn't particularly remarkable compared to other methods. And given the 20th century technology, the deep learning architectures of today were computationally unfeasible.
  • Bartweiss 10y ago
    It does seem rather harsh to hold Minsky to account for a conclusion which is only true with access to the massive computing resources of the 21st century. Not only was the future power of computation unknown in 1955, the quality of neural networks remains nonobvious even with that prediction - you have to actually run the things and see if they work. None of that makes Minsky right, but it's hard to see how much could even have been achieved on neural nets back in '55. Our architecture design today descends from experimental results that were not going to be available for many decades.
    • varjag 10y ago
      True, things often seem simple with advantage of hindsight. However Minsky's original criticism was to linear separability of original perceptron, the only known ANN at the time, and as such as technically sound now as it was then. Even when people got some spare cycles on their computers and started to throw extra layers to increase dimensionality the results weren't too encouraging for a long time. EDIT: well I basically made the same point as you.
      • projectramo 10y ago
        That is true. There are two ways that Minsky could have (mathematically) looked past the linear separability issue: 1. If you add another layer to the perceptron, it can solve the XOR problem. 2. If you add a non-monotonic threshold function, it can solve the XOR problem. So these are two rather simple solutions to the issue he brought up.
        • varjag 10y ago
          Again, hindsight. Backpropagation was first applied to ANNs on the verge of 1980s. So we are slowly moving from Misnky wrongly criticizing perceptrons to Minsky not inventing sound backpropagation/multilayer ANN algorithm, which is perhaps taking it a bit too far.
          • projectramo 10y ago
            Just to be clear, Minsky didn't need backprop, just the multiple layers. (In order for a perceptron to act like an XOR gate, or to "solve the XOR problem" in some parlance.) Minsky assumed a trained percepton with the weights already set to act like an AND or an OR gate. He wasn't dealing with the learning problem.
            • varjag 10y ago
              Fair point, although without knowing that this approach is practically viable to begin with it was getting into speculations territory. Not that Minsky didn't like to speculate.. Anyway, IMO people overestimate Minsky influence to single-handedly shut down an avenue of research. The reason _Perceptrons_ conclusions caught up is because they were sound and reasonable to his peers at the time. I also seem to remember his video interview from a few years back where he elaborates on perceptrons and how much of his original conclusions are applicable to the state of art ANNs. Can't quite find it though.
            • argonaut 10y ago
              To reiterate, people already widely knew multiple layers solved XOR at the time.
  • robrenaud 10y ago
    neural nets were sorting tons of US mail since the 90s.
    • varjag 10y ago
      And symbolic methods beat human players in chess and checkers for decades now, and handled U.S. military logistics since the Desert Storm. Markov chains were writing spamfilter-busting prose for years, and graph clustering powered Google search. It's great to be enthusiastic about breakthroughs, but the history of AI is littered with partial success stories.
• zackmorris 10y ago
  Ya I think so too. I envision strong AI as happening in the next 10-20 years, even as just a random walk of the problem space on clusters of thousands of machines with thousands of cores. But it could take decades after that to get to the point where we understand AI enough to create one with a minimum of cores and algorithms. So he was probably right to discount the importance of creating AI if we don't understand it, but wrong to discount the significance of such an achievement.
• projectramo 10y ago
  True, though he made that decision in a particular historical context. Consider: 1. The computational resources at the time. I tried to run 14 node multi-layer NNs in the 90s and I'd have to go to lunch and come back before a single run was done. (backprop looped several times to converge). 2. The whole symbolic vs statistical debate that was going on at MIT. You had Chomsky, Fodor and Minsky lining up on almost philosophical grounds. (In Fodor's case, explicitly so).