8 ms·

Pursuing the Next Level of Artificial Intelligence

30 points by rockstar9 18y ago

rms 18y ago
>After three decades of disappointments, artificial intelligence researchers are making progress. Do you think AI progress is going to continue accelerating for a while?
- michael_dorfman 18y ago
  I would think so, if for no other reason than Moore's law. As computing power becomes increasingly available, seemingly insignificant steps forward can become significant through the magnification of iteration.
- cia_plant 18y ago
  Old-style AI solved a few of the "smart" things that people do, such as playing chess and proving (simple) mathematical theorems. I predict that new-style AI will solve a few of the "dumb" things that people do, such as keeping our balance and getting a visual sense of our 3d location and orientation. The difficult part will continue to be those areas where we're not just doing combinatorics and we're not just doing statistics. The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks.
  - angstrom 18y ago
    Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages. I don't believe there's any way to solve that without an AI that can actually learn and drive it's own understanding by observing and interacting with its environment both physically and through mimicry.
    - neilc 18y ago
      Why would an AI need to interact with its environment in order to understand language? ISTM that deriving cultural information and understanding language change can both be found by simply observing its environment, and looking at the right corpus (e.g. the Internet).
    - byrneseyeview 18y ago
      "Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages." How much trouble do you have understanding things written in English 100 years ago? 400? 800? Older works require footnotes, but they don't introduce new verbal concepts -- you can footnote Chaucer in modern English, and I suspect that a Chaucer scholar would be able to describe modern concepts by building on old ideas -- there isn't a discontinuity where you have some concept that simply cannot be linked to the words Chaucer would have used. So the problem is not that language changes too fast.
      - aswanson 18y ago
        I think what he is saying is that the generation and extraction of concepts requires context that language may or may not provide. If you think about the approach of AI with language, it is almost the opposite of how humans learn: presentation of grammatics/language first and the extraction of concepts from this, whereas humans learn basic concepts of reality before speaking, and language is a partial representation of what they know.
        byrneseyeview 18y ago
        That's true, but it's not at all what he said. He argued that language is a hard problem because it's linked with culture, and always changing. I think this is incorrect, because it doesn't stop humans from picking up second languages or from understanding older texts in their native languages. The hard part is whatever loop humans have that translates sentences into concepts -- not the loop that uses one sentence-concept chunk to link a new sentence to a new concept. That's in the library, not the interpreter.
  - osipov 18y ago
    >The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks. The most effective existing approaches to natural language understanding are purely statistics based. This works fairly well (about 85% error rate) in narrow domains. Most approaches aren't even bayesian network based, but use Markov chains/models which can be called a precursor to bayesian networks.
    - andreyf 18y ago
      The most effective existing approaches to natural language understanding are purely statistics based. This is where practicality and idealism ought to diverge. Sure, with enough data, we might be able to find spelling mistakes or figure out what pictures someone is looking for. But the techniques we are looking for don't address understanding, (granted they never set out to). Most interesting would be to define meaning and understanding, using strict mathematical models. As McCarty said, language is just the froth on those waves.
- dhs 18y ago
  >Do you think AI progress is going to continue accelerating for a while? From where I stand, it looks like the more recent progress in AI is due to a shift in research hypotheses. It used to be like "AI is the attempted simulation, or sometimes attempted emulation, of human cognition"; now it's more like "AI is a medium for various forms of human expression". Ms. Koller's work emphasizes numerical expression; the work in Game AI emphasizes visual-artistic and/or logical expression, the work in Natural Language Processing emphasizes expression in natural language. Few people try for machines which pass the Turing Test; many try for machines which capture a small slice of real-life semantics and run with it. As Alan Perlis once wrote: "When we write programs that 'learn', it turns out we do and they don't." I believe that we increasingly realize what he meant with this. And that's good, because not only does it say something about what we cannot do, but it also says what we can. People are increasingly aware of this, go pragmatic (Google/MapReduce proved that you can go quite a ways using brute force), and make progress.
- lacker 18y ago
  Yes, we will keep making progress in AI. There's two parts to the success of any machine learning program (like the stuff Daphne Koller is working on mentioned in the article). Your algorithm, and your data. To paraphrase Peter Norvig: "Ten times more data will beat any algorithmic improvement." http://glinden.blogspot.com/2006/10/advantages-of-big-data-and-big.html http://glinden.blogspot.com/2006/10/advantages-of-big-data-a... With the rise of the internet, more and more data will be online. Even if we can't figure out smarter ways of analyzing this data, we will at least keep getting more of it. Here's a thought experiment. What if you had a really large dataset of chat logs, so large that any sequence of 1000 words, you could find it somewhere in the logs. Then you just wrote a chat bot that 1. Looked up the history of the chat in its logs 2. Said whatever the person in the logs said next 3. Once you got to 1000 words it said "Gotta go, talk to you later!" Seems like that could pass the Turing test to me. Of course since there are about a million English words you would need on the order of 1000000 ^ 1000 = a googol^60 items in your data set. ;-)
tokipin 18y ago
is it true most AI was based on logic stuff? no wonder it failed. it's like making a building out of iron legos and wondering why it isn't flexible
- angstrom 18y ago
  For a good read on the subject of AI and how humans think check out the book OnIntelligence by Jeff Hawkins. It covers some about the previous attempts at AI and why they failed. http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/0805074562 http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/080507456...
  - pchristensen 18y ago
    That book is in my top 5 favorite books ever!
    - 9oliYQjP 18y ago
      I keep hearing about this book. Can anybody tell me if it is a pseudo-scientific look at the topic? I don't mean this in a particularly negative way. Rather, what I mean is does it put forth a lot of as-yet unsubstantiated claims perhaps backed up with some hand-waving analysis?
      - pchristensen 18y ago
        Jeff Hawkins (the author, creator of PalmPilot) has always been interested in how the brain works, and once he made his Palm money, he started a neuroscience institute. That research along with his own and what he read led him to his theory of how the brain works. It's extracted from lots of real, hard research. He then started a company (Numenta http://www.numenta.com/ http://www.numenta.com/) to create a software implementation of that theory, which has a set of tools you can download and play with. They've had good results consistent with the theories in the book and had some impressive results. There's also a recording of a talk he gave at an O'Reilly conference: http://itc.conversationsnetwork.org/shows/detail3499.html http://itc.conversationsnetwork.org/shows/detail3499.html So long story short, it is for real. When I read it, I had to stop every 20 minutes or so to catch my breath I found it so exciting and enlightening.
schtog 18y ago
did she invent bayesian spamfilters? i know there was ideas about it before paul graham did it.
- byrneseyeview 18y ago
  http://www.paulgraham.com/better.html http://www.paulgraham.com/better.html The footnotes here cite earlier work, as does the referenced Slashdot article.
pchristensen 18y ago
This is a dupe: http://news.ycombinator.com/item?id=180047 http://news.ycombinator.com/item?id=180047
aswanson 18y ago
Article content is pretty empty, other than saying she uses statistics. Any pointers to her papers?
- plinkplonk 18y ago
  "Article content is pretty empty, other than saying she uses statistics. Any pointers to her papers?" http://ai.stanford.edu/~koller/papers.cgi http://ai.stanford.edu/~koller/papers.cgi enjoy!
  - aswanson 18y ago
    Thanks.
andreyf 18y ago
"My husband still berates me for not having jumped on the Google bandwagon at the beginning," she said. Still, she insists she does not regret her decision to stay in academia. "I like the freedom to explore the things I care about," she said. That seems a bit silly. Given the computational infrastructure available at Google, it would seem that it's the only place where one would be free to try all kinds of ideas unavailable elsewhere. If she's as brilliant as they make her seem, I'm sure they'll let her work on whatever she wants.

30 points by rockstar9 18y ago

rms 18y ago

>After three decades of disappointments, artificial intelligence researchers are making progress. Do you think AI progress is going to continue accelerating for a while?

michael_dorfman 18y ago
I would think so, if for no other reason than Moore's law. As computing power becomes increasingly available, seemingly insignificant steps forward can become significant through the magnification of iteration.
cia_plant 18y ago
Old-style AI solved a few of the "smart" things that people do, such as playing chess and proving (simple) mathematical theorems. I predict that new-style AI will solve a few of the "dumb" things that people do, such as keeping our balance and getting a visual sense of our 3d location and orientation. The difficult part will continue to be those areas where we're not just doing combinatorics and we're not just doing statistics. The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks.
- angstrom 18y ago
  Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages. I don't believe there's any way to solve that without an AI that can actually learn and drive it's own understanding by observing and interacting with its environment both physically and through mimicry.
  - neilc 18y ago
    Why would an AI need to interact with its environment in order to understand language? ISTM that deriving cultural information and understanding language change can both be found by simply observing its environment, and looking at the right corpus (e.g. the Internet).
  - byrneseyeview 18y ago
    "Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages." How much trouble do you have understanding things written in English 100 years ago? 400? 800? Older works require footnotes, but they don't introduce new verbal concepts -- you can footnote Chaucer in modern English, and I suspect that a Chaucer scholar would be able to describe modern concepts by building on old ideas -- there isn't a discontinuity where you have some concept that simply cannot be linked to the words Chaucer would have used. So the problem is not that language changes too fast.
    - aswanson 18y ago
      I think what he is saying is that the generation and extraction of concepts requires context that language may or may not provide. If you think about the approach of AI with language, it is almost the opposite of how humans learn: presentation of grammatics/language first and the extraction of concepts from this, whereas humans learn basic concepts of reality before speaking, and language is a partial representation of what they know.
      - byrneseyeview 18y ago
        That's true, but it's not at all what he said. He argued that language is a hard problem because it's linked with culture, and always changing. I think this is incorrect, because it doesn't stop humans from picking up second languages or from understanding older texts in their native languages. The hard part is whatever loop humans have that translates sentences into concepts -- not the loop that uses one sentence-concept chunk to link a new sentence to a new concept. That's in the library, not the interpreter.
- osipov 18y ago
  >The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks. The most effective existing approaches to natural language understanding are purely statistics based. This works fairly well (about 85% error rate) in narrow domains. Most approaches aren't even bayesian network based, but use Markov chains/models which can be called a precursor to bayesian networks.
  - andreyf 18y ago
    The most effective existing approaches to natural language understanding are purely statistics based. This is where practicality and idealism ought to diverge. Sure, with enough data, we might be able to find spelling mistakes or figure out what pictures someone is looking for. But the techniques we are looking for don't address understanding, (granted they never set out to). Most interesting would be to define meaning and understanding, using strict mathematical models. As McCarty said, language is just the froth on those waves.
dhs 18y ago
>Do you think AI progress is going to continue accelerating for a while? From where I stand, it looks like the more recent progress in AI is due to a shift in research hypotheses. It used to be like "AI is the attempted simulation, or sometimes attempted emulation, of human cognition"; now it's more like "AI is a medium for various forms of human expression". Ms. Koller's work emphasizes numerical expression; the work in Game AI emphasizes visual-artistic and/or logical expression, the work in Natural Language Processing emphasizes expression in natural language. Few people try for machines which pass the Turing Test; many try for machines which capture a small slice of real-life semantics and run with it. As Alan Perlis once wrote: "When we write programs that 'learn', it turns out we do and they don't." I believe that we increasingly realize what he meant with this. And that's good, because not only does it say something about what we cannot do, but it also says what we can. People are increasingly aware of this, go pragmatic (Google/MapReduce proved that you can go quite a ways using brute force), and make progress.
lacker 18y ago
Yes, we will keep making progress in AI. There's two parts to the success of any machine learning program (like the stuff Daphne Koller is working on mentioned in the article). Your algorithm, and your data. To paraphrase Peter Norvig: "Ten times more data will beat any algorithmic improvement." http://glinden.blogspot.com/2006/10/advantages-of-big-data-and-big.html http://glinden.blogspot.com/2006/10/advantages-of-big-data-a... With the rise of the internet, more and more data will be online. Even if we can't figure out smarter ways of analyzing this data, we will at least keep getting more of it. Here's a thought experiment. What if you had a really large dataset of chat logs, so large that any sequence of 1000 words, you could find it somewhere in the logs. Then you just wrote a chat bot that 1. Looked up the history of the chat in its logs 2. Said whatever the person in the logs said next 3. Once you got to 1000 words it said "Gotta go, talk to you later!" Seems like that could pass the Turing test to me. Of course since there are about a million English words you would need on the order of 1000000 ^ 1000 = a googol^60 items in your data set. ;-)

tokipin 18y ago

is it true most AI was based on logic stuff? no wonder it failed. it's like making a building out of iron legos and wondering why it isn't flexible

angstrom 18y ago
For a good read on the subject of AI and how humans think check out the book OnIntelligence by Jeff Hawkins. It covers some about the previous attempts at AI and why they failed. http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/0805074562 http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/080507456...
- pchristensen 18y ago
  That book is in my top 5 favorite books ever!
  - 9oliYQjP 18y ago
    I keep hearing about this book. Can anybody tell me if it is a pseudo-scientific look at the topic? I don't mean this in a particularly negative way. Rather, what I mean is does it put forth a lot of as-yet unsubstantiated claims perhaps backed up with some hand-waving analysis?
    - pchristensen 18y ago
      Jeff Hawkins (the author, creator of PalmPilot) has always been interested in how the brain works, and once he made his Palm money, he started a neuroscience institute. That research along with his own and what he read led him to his theory of how the brain works. It's extracted from lots of real, hard research. He then started a company (Numenta http://www.numenta.com/ http://www.numenta.com/) to create a software implementation of that theory, which has a set of tools you can download and play with. They've had good results consistent with the theories in the book and had some impressive results. There's also a recording of a talk he gave at an O'Reilly conference: http://itc.conversationsnetwork.org/shows/detail3499.html http://itc.conversationsnetwork.org/shows/detail3499.html So long story short, it is for real. When I read it, I had to stop every 20 minutes or so to catch my breath I found it so exciting and enlightening.

schtog 18y ago

did she invent bayesian spamfilters? i know there was ideas about it before paul graham did it.

byrneseyeview 18y ago
http://www.paulgraham.com/better.html http://www.paulgraham.com/better.html The footnotes here cite earlier work, as does the referenced Slashdot article.

pchristensen 18y ago

This is a dupe: http://news.ycombinator.com/item?id=180047 http://news.ycombinator.com/item?id=180047

aswanson 18y ago

Article content is pretty empty, other than saying she uses statistics. Any pointers to her papers?

plinkplonk 18y ago
"Article content is pretty empty, other than saying she uses statistics. Any pointers to her papers?" http://ai.stanford.edu/~koller/papers.cgi http://ai.stanford.edu/~koller/papers.cgi enjoy!
- aswanson 18y ago
  Thanks.

andreyf 18y ago

"My husband still berates me for not having jumped on the Google bandwagon at the beginning," she said. Still, she insists she does not regret her decision to stay in academia. "I like the freedom to explore the things I care about," she said. That seems a bit silly. Given the computational infrastructure available at Google, it would seem that it's the only place where one would be free to try all kinds of ideas unavailable elsewhere. If she's as brilliant as they make her seem, I'm sure they'll let her work on whatever she wants.