9 ms·

Breakthroughs that have unleashed AI development

77 points by ghosh 12y ago

Gabriel_Martin 12y ago
This may be slightly OT, but is it unfair of me to assume that science reporting done by mainstream outlets is at worst so inaccurate and at best half true; to the point that when it involves subjects I know little about, I simply avoid the article entirely, and wait for a reasonably credible source to comment on the matter?
- undersuit 12y ago
  I call that a downvote, or in my current situation no upvote.
- ilaksh 12y ago
  Kevin Kelly is an opponent of the most optimistic AI researchers. For him to use a title like that is a bit of a concession. This is the closest you are going to get to something that is more-or-less accurate and that you can accept. He is reporting that because he has to. Deep learning is not something anyone can dismiss. If Kevin Kelly thought he could downplay the massive importance of deep learning and similar technologies, or say that it wasn't AI, he would. This is actually a fairly pessimistic article in terms of AI prediction, and yet, that title was absolutely justified. AI is here and very powerful, and no one can deny it. People can still pretend, however, that it will never have human-like capabilities, or cannot become smarter than us (at least not in _our_ lifetimes). Give it five to ten years, human-like and beyond-human intelligence will be built, Kevin Kelly will be talking to it, and he will have to write an article about how he was wrong about everything (of course he won't admit that much).
  - quonn 12y ago
    > Give it five to ten years, human-like and beyond-human intelligence will be built It is absolutely safe to say that human-like AI in five years is not going to happen. Even ten years seems highly unlikely. All leading researchers in the field agree on this.
    - PeterisP 12y ago
      If I had a human-level-AI capable machine delivered today to my basement, then it'd still take something like 5 years of training to bring it to a human-level intelligence. Babies are born with all the hardware to have an adult-level intelligence, but converting that potential to actual skills requires months/years for each skill of active learning, experimentation and feedback, not only reading information from the web.
  - michaelkeenan 12y ago
    > Give it five to ten years, human-like and beyond-human intelligence will be built That's probably a bit early. The Machine Intelligence Research Institute has collected various surveys of experts asked about this question. The one with the earliest estimates asked the question, “Assuming beneficial political and economic development and that no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of artificial intelligence that is roughly as good as humans (or better, perhaps unevenly) at science, mathematics, engineering and programming?” Of 19 replies, the median estimates for 10%, 50%, and 90% were 2025, 2035, and 2070, respectively." Other surveys and research are described here: http://intelligence.org/2013/05/15/when-will-ai-be-created/ http://intelligence.org/2013/05/15/when-will-ai-be-created/
    - ilaksh 12y ago
      If you asked those same experts in 2005 if a Jeopardy computer would beat the best human Jeopardy contestants ever in 2011, what percentage of them would go on the record with the correct answer? What is expertise today, tomorrow is passé.
      - kd0amg 12y ago
        Yes, it would have been hard for them to predict that anybody would decide to tackle that particular task. The question from GP is about a much broader range of tasks.
        pinkyand 12y ago
        Watson understands natural language(to some extent).That's a general capability, not a specific task.
        karmacondon 12y ago
        Watson was designed to answer Jeopardy questions. If Alex Trebek had used natural language to ask it anything other than a jeopardy question it would have failed spectacularly. ie, "So where are you from?". It was definitely built for one specific task.
  - pyre 12y ago
    > Give it five to ten years 5-10 years is the default prediction for everything nowadays... We've been 5 - 10 years away from holographic storage mediums for the last 20 years...
- georgemcbay 12y ago
  Not unfair. Also: http://www.goodreads.com/quotes/65213-briefly-stated-the-gell-mann-amnesia-effect-is-as-follows-you http://www.goodreads.com/quotes/65213-briefly-stated-the-gel...
dominotw 12y ago
>OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE I always assumed that breakthroughs in AI would come from philosophy than computer science. We don't even know if have the capability to define the word 'consciousness'.
- ef4 12y ago
  We'll only get new answers to old questions if we're incorporating new inputs or new capabilities. So I would only expect a breakthrough from philosophy if philosophy is integrating new results from computer science (or neuroscience, etc).
  - theoh 12y ago
    If General Relativity could benefit from the application of pre-existing pure mathematics (Riemannian geometry), why shouldn't it be conceivable that computer science could benefit from (possibly ancient) philosophical ideas?
    - eli_gottlieb 12y ago
      Because General Relativity primarily benefited from evidence that a new model was necessary. A-priori thinking can help to expand the space of possible answers, but it's terrible at locating uniquely correct answers within any given space.
      - theoh 12y ago
        I don't accept that premise. It took a while to verify the predictions of General Relativity. It certainly wasn't driven by experimental evidence. Einstein started work on GR with a thought experiment (according to Wikipedia) and of course there's Minkowski's famous observation that SR could have been inferred before Einstein from Galilean Relativity, spatial isotropy and the probable existence of a geometric invariant. In other words, pure math didn't get there, but once they all arrived it could really give insight. Check out Minkowski's famous "staircase wit" comment. Another example from physics is the extent to which Schrodinger was influenced by Eastern philosophy in his formulation of quantum mechanics.
temuze 12y ago
This article says the three breakthroughs were: parallel computation, big data and better algorithms. I think we have a tendency to simplify scientific achievements. There's this romantic notion that real progress happens through eureka moments. Each of these "breakthroughs" are made up of many separate discoveries/inventions that, when aggregated, lead to modern artificial intelligence. So the real question is - what types of new discoveries do we need to keep advancing AI? I'm not an expert in the field, but here are my guesses: - Transfer learning is huge. How can we apply the data used from one problem to solve a different problem? - Generalized pattern matching. Can we use the same algorithm that identifies separate objects in vision to identify different noises? Can we map these problems to the same dimensions? - Better training sets. It takes years for a child to learn object permanence, much less speaking, listening and walking. What data sets can we feed a computer for it to learn about the real, non-virtual world around it? - NLP based off of learning from real world datasets. Can we give a computer data from Google Glass and let it learn edge detection and words and then applying those words to the things it sees? Perhaps progress in NLP will come teaching a computer words the way you would a child instead of hard coding definitions. If you like Wittgenstein, I'd say it's a move away from Tracatus and towards Philosophical Investigations.
thewarrior 12y ago
I'd like to know from AI experts on HN. This is the third wave of AI hype in 50 years. Will this too lead to an AI winter like the last two times or are we really onto something ? Heres an interesting AI company I've found : http://www.celaton.com/solutions http://www.celaton.com/solutions It's already begun.
- Animats 12y ago
  This time there are applications and revenue. In the financial sector, huge amounts of revenue. In the first two "AI revolutions", the startups all went bust. (I went through Stanford CS in 1985, just at the point that it was clear that expert systems were not going to be very useful. Much of the CS faculty back then was in denial about that. More recently, I took a machine learning course. It was taught by someone from Black Rock Capital, not an academic.)
  - sgt101 12y ago
    Interesting - can you describe the curriculum? Or highlight some of the differences of view that you perceived?
    - Animats 12y ago
      The machine learning course, given at Hacker Dojo about three years ago, was right out of Andrew Ng's videos, pre-Coursera, with his awful blackboard handwriting. Support vector machines, that sort of thing. Not enough graphics showing what all those functions are doing. Most of that stuff has a clear graphical representation, but in Ng's original course, you're supposed to imagine it. A Stanford MSCS in the mid-1980s was heavy on mathematical logic. You could get through the whole curriculum without doing a floating point operation. I got discrete math, number theory (Knuth, vol. 2), proof of correctness, formal logic, proof of correctness, theory of programming languages, etc. Some graphics work, on a Xerox Alto. Dr. John's Mystery Hour, "Epistemological Problems in Artificial Intelligence", from John McCarthy. (He would describe a problem informally, then a miracle occurs, and it's in a predicate calculus notation where you just turn the crank to get the answer. It hadn't yet hit the logicians that getting the problem into the formalism is the hard part. Automatic symbolic math was in its infancy back then.)
slackstation 12y ago
"AS AIS DEVELOP, WE MIGHT HAVE TO ENGINEER WAYS TO PREVENT CONSCIOUSNESS IN THEM—OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE." No. Simply, no. We don't understand consciousness well enough in our own minds to understand how to stop that mechanism happening in another mind. As of yet, we have no mechanism that produces any of the insightful, creative, general intelligence that we see in humans. Even basic biological processes like walking over uneven terrain, flight by flapping of wings and picking up novel and oddly-shaped objects that haven't been seen before are challenges that we haven't even begun to master. The hyperbole of these articles makes it seem as if creative machine intelligence is right around the corner. What we have done is make statistical pattern matching algorithms. They aren't learning in the way that a child learns through repetition. We simply don't know how general intelligence works well enough to do this.
- mortenjorck 12y ago
  I had to stop reading after that line. “Our most premium services will be advertised as consciousness-free” literally makes as much sense as a pre-Wright-Brothers article predicting that “our most premium flying machines will be advertised as free of feathers.” It’s just a non-sequitur given a grasp of the basic principles, whether it be flight or AI.
- cpeterso 12y ago
  > OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE. I misread this as: "our most premium ad services will be advertised as conscience-free." :)
- ilaksh 12y ago
  https://www.youtube.com/watch?v=XBsl3HlB8VE https://www.youtube.com/watch?v=XBsl3HlB8VE Insight, creativity, and general intelligence are all different and active areas of research. There is quite a lot of progress in creativity. Google for 'creative software'. Insight can mean many things, but Watson can now provide insights into cancer diagnosis. Consciousness is actually fairly well understood in terms of attention, focus, and other aspects. Artificial general intelligence is a very active field seeing quite a bit of progress. Here is a bird that flies with flapping wings: https://www.indiegogo.com/projects/bionic-bird-the-flying-app--2 https://www.indiegogo.com/projects/bionic-bird-the-flying-ap... (By the way, that is completely unrelated to AI). Walking over uneven terrain, (also completely unrelated to artificial general intelligence): https://www.youtube.com/watch?v=uVG4J29JZI0 https://www.youtube.com/watch?v=uVG4J29JZI0 https://www.youtube.com/watch?v=W1czBcnX1Ww https://www.youtube.com/watch?v=W1czBcnX1Ww Deep learning is beyond statistical pattern matching. It does involve both supervised and unsupervised learning. Deep learning is currently the most successful technique, but not the most ambitious approach in AGI. Google for 'AGI', 'deep learning', 'sparse autoencoder', 'hierarchical hidden Markov model', 'OpenCog', 'spiking neural network', 'Hierarchical Temporal Memory'
kleiba 12y ago
The Three Breakthroughs That Have Finally Unleashed AI on the World [...] 3. Better algorithms To be honest, I find the article a bit missing because of insights like this.
ha292 12y ago
The article is almost content free. Maybe it was written via an AI-tool.

77 points by ghosh 12y ago

This may be slightly OT, but is it unfair of me to assume that science reporting done by mainstream outlets is at worst so inaccurate and at best half true; to the point that when it involves subjects I know little about, I simply avoid the article entirely, and wait for a reasonably credible source to comment on the matter?

undersuit 12y ago
I call that a downvote, or in my current situation no upvote.
ilaksh 12y ago
Kevin Kelly is an opponent of the most optimistic AI researchers. For him to use a title like that is a bit of a concession. This is the closest you are going to get to something that is more-or-less accurate and that you can accept. He is reporting that because he has to. Deep learning is not something anyone can dismiss. If Kevin Kelly thought he could downplay the massive importance of deep learning and similar technologies, or say that it wasn't AI, he would. This is actually a fairly pessimistic article in terms of AI prediction, and yet, that title was absolutely justified. AI is here and very powerful, and no one can deny it. People can still pretend, however, that it will never have human-like capabilities, or cannot become smarter than us (at least not in _our_ lifetimes). Give it five to ten years, human-like and beyond-human intelligence will be built, Kevin Kelly will be talking to it, and he will have to write an article about how he was wrong about everything (of course he won't admit that much).
- quonn 12y ago
  > Give it five to ten years, human-like and beyond-human intelligence will be built It is absolutely safe to say that human-like AI in five years is not going to happen. Even ten years seems highly unlikely. All leading researchers in the field agree on this.
  - PeterisP 12y ago
    If I had a human-level-AI capable machine delivered today to my basement, then it'd still take something like 5 years of training to bring it to a human-level intelligence. Babies are born with all the hardware to have an adult-level intelligence, but converting that potential to actual skills requires months/years for each skill of active learning, experimentation and feedback, not only reading information from the web.
- michaelkeenan 12y ago
  > Give it five to ten years, human-like and beyond-human intelligence will be built That's probably a bit early. The Machine Intelligence Research Institute has collected various surveys of experts asked about this question. The one with the earliest estimates asked the question, “Assuming beneficial political and economic development and that no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of artificial intelligence that is roughly as good as humans (or better, perhaps unevenly) at science, mathematics, engineering and programming?” Of 19 replies, the median estimates for 10%, 50%, and 90% were 2025, 2035, and 2070, respectively." Other surveys and research are described here: http://intelligence.org/2013/05/15/when-will-ai-be-created/ http://intelligence.org/2013/05/15/when-will-ai-be-created/
  - ilaksh 12y ago
    If you asked those same experts in 2005 if a Jeopardy computer would beat the best human Jeopardy contestants ever in 2011, what percentage of them would go on the record with the correct answer? What is expertise today, tomorrow is passé.
    - kd0amg 12y ago
      Yes, it would have been hard for them to predict that anybody would decide to tackle that particular task. The question from GP is about a much broader range of tasks.
      - pinkyand 12y ago
        Watson understands natural language(to some extent).That's a general capability, not a specific task.
        karmacondon 12y ago
        Watson was designed to answer Jeopardy questions. If Alex Trebek had used natural language to ask it anything other than a jeopardy question it would have failed spectacularly. ie, "So where are you from?". It was definitely built for one specific task.
- pyre 12y ago
  > Give it five to ten years 5-10 years is the default prediction for everything nowadays... We've been 5 - 10 years away from holographic storage mediums for the last 20 years...
georgemcbay 12y ago
Not unfair. Also: http://www.goodreads.com/quotes/65213-briefly-stated-the-gell-mann-amnesia-effect-is-as-follows-you http://www.goodreads.com/quotes/65213-briefly-stated-the-gel...

dominotw 12y ago

>OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE I always assumed that breakthroughs in AI would come from philosophy than computer science. We don't even know if have the capability to define the word 'consciousness'.

ef4 12y ago
We'll only get new answers to old questions if we're incorporating new inputs or new capabilities. So I would only expect a breakthrough from philosophy if philosophy is integrating new results from computer science (or neuroscience, etc).
- theoh 12y ago
  If General Relativity could benefit from the application of pre-existing pure mathematics (Riemannian geometry), why shouldn't it be conceivable that computer science could benefit from (possibly ancient) philosophical ideas?
  - eli_gottlieb 12y ago
    Because General Relativity primarily benefited from evidence that a new model was necessary. A-priori thinking can help to expand the space of possible answers, but it's terrible at locating uniquely correct answers within any given space.
    - theoh 12y ago
      I don't accept that premise. It took a while to verify the predictions of General Relativity. It certainly wasn't driven by experimental evidence. Einstein started work on GR with a thought experiment (according to Wikipedia) and of course there's Minkowski's famous observation that SR could have been inferred before Einstein from Galilean Relativity, spatial isotropy and the probable existence of a geometric invariant. In other words, pure math didn't get there, but once they all arrived it could really give insight. Check out Minkowski's famous "staircase wit" comment. Another example from physics is the extent to which Schrodinger was influenced by Eastern philosophy in his formulation of quantum mechanics.

temuze 12y ago

This article says the three breakthroughs were: parallel computation, big data and better algorithms. I think we have a tendency to simplify scientific achievements. There's this romantic notion that real progress happens through eureka moments. Each of these "breakthroughs" are made up of many separate discoveries/inventions that, when aggregated, lead to modern artificial intelligence. So the real question is - what types of new discoveries do we need to keep advancing AI? I'm not an expert in the field, but here are my guesses: - Transfer learning is huge. How can we apply the data used from one problem to solve a different problem? - Generalized pattern matching. Can we use the same algorithm that identifies separate objects in vision to identify different noises? Can we map these problems to the same dimensions? - Better training sets. It takes years for a child to learn object permanence, much less speaking, listening and walking. What data sets can we feed a computer for it to learn about the real, non-virtual world around it? - NLP based off of learning from real world datasets. Can we give a computer data from Google Glass and let it learn edge detection and words and then applying those words to the things it sees? Perhaps progress in NLP will come teaching a computer words the way you would a child instead of hard coding definitions. If you like Wittgenstein, I'd say it's a move away from Tracatus and towards Philosophical Investigations.

thewarrior 12y ago

I'd like to know from AI experts on HN. This is the third wave of AI hype in 50 years. Will this too lead to an AI winter like the last two times or are we really onto something ? Heres an interesting AI company I've found : http://www.celaton.com/solutions http://www.celaton.com/solutions It's already begun.

Animats 12y ago
This time there are applications and revenue. In the financial sector, huge amounts of revenue. In the first two "AI revolutions", the startups all went bust. (I went through Stanford CS in 1985, just at the point that it was clear that expert systems were not going to be very useful. Much of the CS faculty back then was in denial about that. More recently, I took a machine learning course. It was taught by someone from Black Rock Capital, not an academic.)
- sgt101 12y ago
  Interesting - can you describe the curriculum? Or highlight some of the differences of view that you perceived?
  - Animats 12y ago
    The machine learning course, given at Hacker Dojo about three years ago, was right out of Andrew Ng's videos, pre-Coursera, with his awful blackboard handwriting. Support vector machines, that sort of thing. Not enough graphics showing what all those functions are doing. Most of that stuff has a clear graphical representation, but in Ng's original course, you're supposed to imagine it. A Stanford MSCS in the mid-1980s was heavy on mathematical logic. You could get through the whole curriculum without doing a floating point operation. I got discrete math, number theory (Knuth, vol. 2), proof of correctness, formal logic, proof of correctness, theory of programming languages, etc. Some graphics work, on a Xerox Alto. Dr. John's Mystery Hour, "Epistemological Problems in Artificial Intelligence", from John McCarthy. (He would describe a problem informally, then a miracle occurs, and it's in a predicate calculus notation where you just turn the crank to get the answer. It hadn't yet hit the logicians that getting the problem into the formalism is the hard part. Automatic symbolic math was in its infancy back then.)

slackstation 12y ago

"AS AIS DEVELOP, WE MIGHT HAVE TO ENGINEER WAYS TO PREVENT CONSCIOUSNESS IN THEM—OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE." No. Simply, no. We don't understand consciousness well enough in our own minds to understand how to stop that mechanism happening in another mind. As of yet, we have no mechanism that produces any of the insightful, creative, general intelligence that we see in humans. Even basic biological processes like walking over uneven terrain, flight by flapping of wings and picking up novel and oddly-shaped objects that haven't been seen before are challenges that we haven't even begun to master. The hyperbole of these articles makes it seem as if creative machine intelligence is right around the corner. What we have done is make statistical pattern matching algorithms. They aren't learning in the way that a child learns through repetition. We simply don't know how general intelligence works well enough to do this.

mortenjorck 12y ago
I had to stop reading after that line. “Our most premium services will be advertised as consciousness-free” literally makes as much sense as a pre-Wright-Brothers article predicting that “our most premium flying machines will be advertised as free of feathers.” It’s just a non-sequitur given a grasp of the basic principles, whether it be flight or AI.
cpeterso 12y ago
> OUR MOST PREMIUM AI SERVICES WILL BE ADVERTISED AS CONSCIOUSNESS-FREE. I misread this as: "our most premium ad services will be advertised as conscience-free." :)
ilaksh 12y ago
https://www.youtube.com/watch?v=XBsl3HlB8VE https://www.youtube.com/watch?v=XBsl3HlB8VE Insight, creativity, and general intelligence are all different and active areas of research. There is quite a lot of progress in creativity. Google for 'creative software'. Insight can mean many things, but Watson can now provide insights into cancer diagnosis. Consciousness is actually fairly well understood in terms of attention, focus, and other aspects. Artificial general intelligence is a very active field seeing quite a bit of progress. Here is a bird that flies with flapping wings: https://www.indiegogo.com/projects/bionic-bird-the-flying-app--2 https://www.indiegogo.com/projects/bionic-bird-the-flying-ap... (By the way, that is completely unrelated to AI). Walking over uneven terrain, (also completely unrelated to artificial general intelligence): https://www.youtube.com/watch?v=uVG4J29JZI0 https://www.youtube.com/watch?v=uVG4J29JZI0 https://www.youtube.com/watch?v=W1czBcnX1Ww https://www.youtube.com/watch?v=W1czBcnX1Ww Deep learning is beyond statistical pattern matching. It does involve both supervised and unsupervised learning. Deep learning is currently the most successful technique, but not the most ambitious approach in AGI. Google for 'AGI', 'deep learning', 'sparse autoencoder', 'hierarchical hidden Markov model', 'OpenCog', 'spiking neural network', 'Hierarchical Temporal Memory'

kleiba 12y ago

The Three Breakthroughs That Have Finally Unleashed AI on the World [...] 3. Better algorithms To be honest, I find the article a bit missing because of insights like this.

ha292 12y ago

The article is almost content free. Maybe it was written via an AI-tool.