5 ms·

>For more than 30 years, Geoffrey Hinton hovered at the edges of artificial intelligence research, an outsider clinging to a simple proposition: that computers

by rm999 9y ago

>For more than 30 years, Geoffrey Hinton hovered at the edges of artificial intelligence research, an outsider clinging to a simple proposition: that computers could think like humans do—using intuition rather than rules. This is so disrespectful to the 1000s of researchers who have been studying machine learning since well before 2012. It was well established that the future of teaching computers came from statistics and not rules in the 80s/90s by researchers like Michael Jordan (https://en.wikipedia.org/wiki/Michael_I._Jordan https://en.wikipedia.org/wiki/Michael_I._Jordan) and his students. It was even engrained in popular culture: neural networks are how the AI brain worked in Terminator 2, in 1991! https://www.youtube.com/watch?v=xcgVztdMrX4 https://www.youtube.com/watch?v=xcgVztdMrX4 edit: I don't want to downplay Hinton's accomplishments, I've been lucky to have been surrounded by and motivated by his work since I started learning machine learning. I did my masters research on neural networks that were partly inspired by his work, and it was a deep networks paper he presented at a NIPS 2006 workshop that got me really excited to stay in machine learning while I was starting my career.

justicezyx 9y ago
Winner takes all, that's how human brain works... Thats why I am fundamentally pessimistic about human future. ..
nostrademons 9y ago
There've been cycles & fads in the meantime, though. I remember that in the early 90s, neural nets were supposed to be the huge new thing. Many scanners shipped with built-in OCR, the Apple Newton had handwriting recognition on a PDA, and my Centris 660AV could do speech recognition & text-to-speech out of the box. But they ultimately weren't powerful enough to satisfy customers or meaningfully change how people interacted with computers, so they failed in the market, and the hype cycle moved on to the World Wide Web. I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits.
- jstarfish 9y ago
  > I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits. So true. VR popped up in the 90/00s (Virtual Boy, VRML, etc.), went nowhere, and here we are again. I suspect something similar will happen with blockchain/cryptocurrency. Only once all the hype and speculation dies off will meaningful uses for the tech become evident.
  - alanfalcon 9y ago
    Are you trying to suggest that Cryptokitties aren’t meaningful? Blasphemy :-)
- olympus 9y ago
  The hard part is surviving the 20 year drought of research funding in the meantime. Most researchers want to hang onto an idea and develop it until it changes the world but they need to keep their job, so they follow the funding and that means following the fads.
  - jamesblonde 9y ago
    So true. I did part of my PhD on reinforcement learning, finished in 2004. I couldn't get money for continuing work on the topic (wasn't cool), so ended up getting into P2P and Big Data. Still do a bit of ML, but it's not core anymore.
deleted 9y ago
[deleted]
kahnjw 9y ago
Could not have said it better. Hinton is one of the most important contributors to the field, but giving him all the credit is a disservice to others who made important contributions. Probabilistic modeling was popular way before DNNs got big. At that point it was already clear that logic based inference was only so powerful, and different methods were necessary.
- YeGoblynQueenne 9y ago
  >> At that point it was already clear that logic based inference was only so powerful, and different methods were necessary. Er, logic inference is "only so powerful"? The first order predicate calculus (first order logic) is Turing-complete and there is plenty of maths that prove the soundess and completeness of various logic inference rules. In other words, if you can compute a function, you can compute it with first order logic. The switch from symbolic to statistical AI happened partly because of the AI winter that cut the funding to all AI, which at the time was primarily symbolic AI, partly because it became evident that developing and maintaining huge databases of logic rules was inefficient. Some of the early work in machine learning focused on overcoming this inefficiency by inducing rules from data.
  - kahnjw 9y ago
    >> The first order predicate calculus (first order logic) is Turing-complete Brainfuck is turing complete, does that make it as powerful as a general purpose programming language such as Java? I'm not criticizing first order logic, it has use cases no doubt. On the other hand there are many cases where codifying knowledge into a we formed KB isn't practical. Some of these use cases are better suited to probabilistic approaches, deep learning, and some are unsolved. That pretty clearly puts limits on the power of FOL. Also, thanks for mansplaining turing completeness.
    - YeGoblynQueenne 9y ago
      >> Also, thanks for mansplaining turing completeness. Snark is not OK: Be civil. Don't say things you wouldn't say face-to-face. Don't be snarky. Comments should get more civil and substantive, not less, as a topic gets more divisive. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
deleted 9y ago
[deleted]
happy-go-lucky 9y ago
I see the article just as an effort at highlighting Hinton and his AI research. I don't think he would say he had done it all. Toronto Life is saying it all. I have just finished reading this interesting wiki titled AI winter defined as a period of reduced funding and interest in AI research: https://en.m.wikipedia.org/wiki/AI_winter https://en.m.wikipedia.org/wiki/AI_winter There have been such winters since the 60's.
aje403 9y ago
We get blasted with X greatest sports player of all time, or Y most important historical figure! None of those are really accurate, but are those people important and accomplished enough that they deserve someone lauding them so highly along with the footnote in history they receive? Whether or not they deserve the prestige they're receiving or sole credit for whatever accomplishment? I am not disputing whether or not the article's portrayal is entirely accurate.
chriskanan 9y ago
While I won't dispute that many in AI thought machine learning was the correct path, the majority of researchers in machine learning in the 2000s were dismissive of neural networks as the way forward. When I started my PhD in 2007 with the specific goal of studying deep learning, many of my peers told me this was a very bad career move, because neural networks were a dead technology. Convex optimization, kernels, and variational Bayesian methods were the future. They were wrong. AI has had a lot of fads, but neural networks are here to stay, in my opinion.
- robotresearcher 9y ago
  Convex optimization, kernels and Bayesian methods are also here to stay. They are all in the toolbox. Optimization-based methods are still rapidly spreading in robotics, for example, even as we add CNNs to the mix.
- shaklee3 9y ago
  Convex optimization is heavily used in industry for real problems. It just doesn't have the sexiness that neural networks too, so it doesn't get as much attention.
  - chriskanan 9y ago
    I agree. My comment about being wrong was regarding neural networks being a dead end. Convex optimization and Bayesian techniques are both incredibly useful.
deleted 9y ago
[deleted]
tzs 9y ago
Didn't Hinton stick with neural networks as his primary (often only?) research focus for all that time, whereas most of those other 1000s of researchers moved in and out of neural network research as the popularity of neural networks waxed and waned? I think that is what the article may be getting it.
darepublic 9y ago
In respect to the first quote from the article "computers could think like humans do -- using intuition rather than rules". Is this really a good description of neural networks? I don't think so, to me it sounds like a bad misrepresentation of what neural networks are by a writer who can't describe the way ai works without humanizing it inaccurately.
- YeGoblynQueenne 9y ago
  That's probably Hinton's words.
  - darepublic 9y ago
    If that is the case I am eating my words
YeGoblynQueenne 9y ago
>> It was well established that the future of teaching computers came from statistics and not rules in the 80s/90s by researchers like Michael Jordan (https://en.wikipedia.org/wiki/Michael_I._Jordan https://en.wikipedia.org/wiki/Michael_I._Jordan) and his students. As a counterpoint, one of the most successful (classes of) machine learning algorithms are Decision Tree learners, whose models are decidedly symbolic. There's also plenty of work on learning first-order logic theories with neural nets (see for instance the work of Artur D'Avilla Garcez). The problem with rules is that it's hard to develop and maintain large rule-bases. However, it's perfectly possible to do machine learning for rules -and logic-based machine learning is totally a thing (full disclosure: it's the thing I'm doing a PhD on). You probably haven't heard of it though because data scientists tend to be good with statistics but bad with symbolic logic.

by rm999 9y ago

justicezyx 9y ago

Winner takes all, that's how human brain works... Thats why I am fundamentally pessimistic about human future. ..

nostrademons 9y ago

There've been cycles & fads in the meantime, though. I remember that in the early 90s, neural nets were supposed to be the huge new thing. Many scanners shipped with built-in OCR, the Apple Newton had handwriting recognition on a PDA, and my Centris 660AV could do speech recognition & text-to-speech out of the box. But they ultimately weren't powerful enough to satisfy customers or meaningfully change how people interacted with computers, so they failed in the market, and the hype cycle moved on to the World Wide Web. I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits.

jstarfish 9y ago
> I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits. So true. VR popped up in the 90/00s (Virtual Boy, VRML, etc.), went nowhere, and here we are again. I suspect something similar will happen with blockchain/cryptocurrency. Only once all the hype and speculation dies off will meaningful uses for the tech become evident.
- alanfalcon 9y ago
  Are you trying to suggest that Cryptokitties aren’t meaningful? Blasphemy :-)
olympus 9y ago
The hard part is surviving the 20 year drought of research funding in the meantime. Most researchers want to hang onto an idea and develop it until it changes the world but they need to keep their job, so they follow the funding and that means following the fads.
- jamesblonde 9y ago
  So true. I did part of my PhD on reinforcement learning, finished in 2004. I couldn't get money for continuing work on the topic (wasn't cool), so ended up getting into P2P and Big Data. Still do a bit of ML, but it's not core anymore.

deleted 9y ago

[deleted]

kahnjw 9y ago

Could not have said it better. Hinton is one of the most important contributors to the field, but giving him all the credit is a disservice to others who made important contributions. Probabilistic modeling was popular way before DNNs got big. At that point it was already clear that logic based inference was only so powerful, and different methods were necessary.

YeGoblynQueenne 9y ago
>> At that point it was already clear that logic based inference was only so powerful, and different methods were necessary. Er, logic inference is "only so powerful"? The first order predicate calculus (first order logic) is Turing-complete and there is plenty of maths that prove the soundess and completeness of various logic inference rules. In other words, if you can compute a function, you can compute it with first order logic. The switch from symbolic to statistical AI happened partly because of the AI winter that cut the funding to all AI, which at the time was primarily symbolic AI, partly because it became evident that developing and maintaining huge databases of logic rules was inefficient. Some of the early work in machine learning focused on overcoming this inefficiency by inducing rules from data.
- kahnjw 9y ago
  >> The first order predicate calculus (first order logic) is Turing-complete Brainfuck is turing complete, does that make it as powerful as a general purpose programming language such as Java? I'm not criticizing first order logic, it has use cases no doubt. On the other hand there are many cases where codifying knowledge into a we formed KB isn't practical. Some of these use cases are better suited to probabilistic approaches, deep learning, and some are unsolved. That pretty clearly puts limits on the power of FOL. Also, thanks for mansplaining turing completeness.
  - YeGoblynQueenne 9y ago
    >> Also, thanks for mansplaining turing completeness. Snark is not OK: Be civil. Don't say things you wouldn't say face-to-face. Don't be snarky. Comments should get more civil and substantive, not less, as a topic gets more divisive. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

deleted 9y ago

[deleted]

happy-go-lucky 9y ago

I see the article just as an effort at highlighting Hinton and his AI research. I don't think he would say he had done it all. Toronto Life is saying it all. I have just finished reading this interesting wiki titled AI winter defined as a period of reduced funding and interest in AI research: https://en.m.wikipedia.org/wiki/AI_winter https://en.m.wikipedia.org/wiki/AI_winter There have been such winters since the 60's.

aje403 9y ago

We get blasted with X greatest sports player of all time, or Y most important historical figure! None of those are really accurate, but are those people important and accomplished enough that they deserve someone lauding them so highly along with the footnote in history they receive? Whether or not they deserve the prestige they're receiving or sole credit for whatever accomplishment? I am not disputing whether or not the article's portrayal is entirely accurate.

chriskanan 9y ago

While I won't dispute that many in AI thought machine learning was the correct path, the majority of researchers in machine learning in the 2000s were dismissive of neural networks as the way forward. When I started my PhD in 2007 with the specific goal of studying deep learning, many of my peers told me this was a very bad career move, because neural networks were a dead technology. Convex optimization, kernels, and variational Bayesian methods were the future. They were wrong. AI has had a lot of fads, but neural networks are here to stay, in my opinion.

robotresearcher 9y ago
Convex optimization, kernels and Bayesian methods are also here to stay. They are all in the toolbox. Optimization-based methods are still rapidly spreading in robotics, for example, even as we add CNNs to the mix.
shaklee3 9y ago
Convex optimization is heavily used in industry for real problems. It just doesn't have the sexiness that neural networks too, so it doesn't get as much attention.
- chriskanan 9y ago
  I agree. My comment about being wrong was regarding neural networks being a dead end. Convex optimization and Bayesian techniques are both incredibly useful.

deleted 9y ago

[deleted]

tzs 9y ago

Didn't Hinton stick with neural networks as his primary (often only?) research focus for all that time, whereas most of those other 1000s of researchers moved in and out of neural network research as the popularity of neural networks waxed and waned? I think that is what the article may be getting it.

darepublic 9y ago

In respect to the first quote from the article "computers could think like humans do -- using intuition rather than rules". Is this really a good description of neural networks? I don't think so, to me it sounds like a bad misrepresentation of what neural networks are by a writer who can't describe the way ai works without humanizing it inaccurately.

YeGoblynQueenne 9y ago
That's probably Hinton's words.
- darepublic 9y ago
  If that is the case I am eating my words

YeGoblynQueenne 9y ago

>> It was well established that the future of teaching computers came from statistics and not rules in the 80s/90s by researchers like Michael Jordan (https://en.wikipedia.org/wiki/Michael_I._Jordan https://en.wikipedia.org/wiki/Michael_I._Jordan) and his students. As a counterpoint, one of the most successful (classes of) machine learning algorithms are Decision Tree learners, whose models are decidedly symbolic. There's also plenty of work on learning first-order logic theories with neural nets (see for instance the work of Artur D'Avilla Garcez). The problem with rules is that it's hard to develop and maintain large rule-bases. However, it's perfectly possible to do machine learning for rules -and logic-based machine learning is totally a thing (full disclosure: it's the thing I'm doing a PhD on). You probably haven't heard of it though because data scientists tend to be good with statistics but bad with symbolic logic.