4 ms·

>After three decades of disappointments, artificial intelligence researchers are making progress. Do you think AI progress is going to continue accelerating fo

by rms 19y ago

>After three decades of disappointments, artificial intelligence researchers are making progress. Do you think AI progress is going to continue accelerating for a while?

michael_dorfman 19y ago
I would think so, if for no other reason than Moore's law. As computing power becomes increasingly available, seemingly insignificant steps forward can become significant through the magnification of iteration.
cia_plant 19y ago
Old-style AI solved a few of the "smart" things that people do, such as playing chess and proving (simple) mathematical theorems. I predict that new-style AI will solve a few of the "dumb" things that people do, such as keeping our balance and getting a visual sense of our 3d location and orientation. The difficult part will continue to be those areas where we're not just doing combinatorics and we're not just doing statistics. The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks.
- angstrom 19y ago
  Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages. I don't believe there's any way to solve that without an AI that can actually learn and drive it's own understanding by observing and interacting with its environment both physically and through mimicry.
  - neilc 19y ago
    Why would an AI need to interact with its environment in order to understand language? ISTM that deriving cultural information and understanding language change can both be found by simply observing its environment, and looking at the right corpus (e.g. the Internet).
  - byrneseyeview 19y ago
    "Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages." How much trouble do you have understanding things written in English 100 years ago? 400? 800? Older works require footnotes, but they don't introduce new verbal concepts -- you can footnote Chaucer in modern English, and I suspect that a Chaucer scholar would be able to describe modern concepts by building on old ideas -- there isn't a discontinuity where you have some concept that simply cannot be linked to the words Chaucer would have used. So the problem is not that language changes too fast.
    - aswanson 19y ago
      I think what he is saying is that the generation and extraction of concepts requires context that language may or may not provide. If you think about the approach of AI with language, it is almost the opposite of how humans learn: presentation of grammatics/language first and the extraction of concepts from this, whereas humans learn basic concepts of reality before speaking, and language is a partial representation of what they know.
      - byrneseyeview 19y ago
        That's true, but it's not at all what he said. He argued that language is a hard problem because it's linked with culture, and always changing. I think this is incorrect, because it doesn't stop humans from picking up second languages or from understanding older texts in their native languages. The hard part is whatever loop humans have that translates sentences into concepts -- not the loop that uses one sentence-concept chunk to link a new sentence to a new concept. That's in the library, not the interpreter.
        aswanson 19y ago
        That's true, but it's not at all what he said What he implied from culture, I took to mean a polymorphic mapping of "sentences" or "units of communication" in a given understood context that can be learned only through experience. it doesn't stop humans from picking up second languages or from understanding older texts in their native languages It doesn't stop them, but it is far from trivial to learn a new language or to even resolve syntactic,lexical, or sematic ambiguity without context, even in a native speakers language: http://en.wikipedia.org/wiki/Ambiguity http://en.wikipedia.org/wiki/Ambiguity. The ill-posed nature of this mapping is not static, and for people not speaking their native language or young children this becomes the basis for some forms of comedy. As far as I know we have no generic grammar that can generate the rules for all languages; there is no universal "interpreter", and it is not certain that "concept" can be decoupled from "phrase" or "sentence" without context, so I don't think your separation of "library" and "interpreter" holds.
- osipov 19y ago
  >The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks. The most effective existing approaches to natural language understanding are purely statistics based. This works fairly well (about 85% error rate) in narrow domains. Most approaches aren't even bayesian network based, but use Markov chains/models which can be called a precursor to bayesian networks.
  - andreyf 19y ago
    The most effective existing approaches to natural language understanding are purely statistics based. This is where practicality and idealism ought to diverge. Sure, with enough data, we might be able to find spelling mistakes or figure out what pictures someone is looking for. But the techniques we are looking for don't address understanding, (granted they never set out to). Most interesting would be to define meaning and understanding, using strict mathematical models. As McCarty said, language is just the froth on those waves.
dhs 19y ago
>Do you think AI progress is going to continue accelerating for a while? From where I stand, it looks like the more recent progress in AI is due to a shift in research hypotheses. It used to be like "AI is the attempted simulation, or sometimes attempted emulation, of human cognition"; now it's more like "AI is a medium for various forms of human expression". Ms. Koller's work emphasizes numerical expression; the work in Game AI emphasizes visual-artistic and/or logical expression, the work in Natural Language Processing emphasizes expression in natural language. Few people try for machines which pass the Turing Test; many try for machines which capture a small slice of real-life semantics and run with it. As Alan Perlis once wrote: "When we write programs that 'learn', it turns out we do and they don't." I believe that we increasingly realize what he meant with this. And that's good, because not only does it say something about what we cannot do, but it also says what we can. People are increasingly aware of this, go pragmatic (Google/MapReduce proved that you can go quite a ways using brute force), and make progress.
lacker 19y ago
Yes, we will keep making progress in AI. There's two parts to the success of any machine learning program (like the stuff Daphne Koller is working on mentioned in the article). Your algorithm, and your data. To paraphrase Peter Norvig: "Ten times more data will beat any algorithmic improvement." http://glinden.blogspot.com/2006/10/advantages-of-big-data-and-big.html http://glinden.blogspot.com/2006/10/advantages-of-big-data-a... With the rise of the internet, more and more data will be online. Even if we can't figure out smarter ways of analyzing this data, we will at least keep getting more of it. Here's a thought experiment. What if you had a really large dataset of chat logs, so large that any sequence of 1000 words, you could find it somewhere in the logs. Then you just wrote a chat bot that 1. Looked up the history of the chat in its logs 2. Said whatever the person in the logs said next 3. Once you got to 1000 words it said "Gotta go, talk to you later!" Seems like that could pass the Turing test to me. Of course since there are about a million English words you would need on the order of 1000000 ^ 1000 = a googol^60 items in your data set. ;-)

by rms 19y ago

>After three decades of disappointments, artificial intelligence researchers are making progress. Do you think AI progress is going to continue accelerating for a while?

michael_dorfman 19y ago

I would think so, if for no other reason than Moore's law. As computing power becomes increasingly available, seemingly insignificant steps forward can become significant through the magnification of iteration.

cia_plant 19y ago

Old-style AI solved a few of the "smart" things that people do, such as playing chess and proving (simple) mathematical theorems. I predict that new-style AI will solve a few of the "dumb" things that people do, such as keeping our balance and getting a visual sense of our 3d location and orientation. The difficult part will continue to be those areas where we're not just doing combinatorics and we're not just doing statistics. The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks.

angstrom 19y ago
Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages. I don't believe there's any way to solve that without an AI that can actually learn and drive it's own understanding by observing and interacting with its environment both physically and through mimicry.
- neilc 19y ago
  Why would an AI need to interact with its environment in order to understand language? ISTM that deriving cultural information and understanding language change can both be found by simply observing its environment, and looking at the right corpus (e.g. the Internet).
- byrneseyeview 19y ago
  "Language will be difficult for quite some time due to its links with culture and the fact that it is always evolving and cross polinating between languages." How much trouble do you have understanding things written in English 100 years ago? 400? 800? Older works require footnotes, but they don't introduce new verbal concepts -- you can footnote Chaucer in modern English, and I suspect that a Chaucer scholar would be able to describe modern concepts by building on old ideas -- there isn't a discontinuity where you have some concept that simply cannot be linked to the words Chaucer would have used. So the problem is not that language changes too fast.
  - aswanson 19y ago
    I think what he is saying is that the generation and extraction of concepts requires context that language may or may not provide. If you think about the approach of AI with language, it is almost the opposite of how humans learn: presentation of grammatics/language first and the extraction of concepts from this, whereas humans learn basic concepts of reality before speaking, and language is a partial representation of what they know.
    - byrneseyeview 19y ago
      That's true, but it's not at all what he said. He argued that language is a hard problem because it's linked with culture, and always changing. I think this is incorrect, because it doesn't stop humans from picking up second languages or from understanding older texts in their native languages. The hard part is whatever loop humans have that translates sentences into concepts -- not the loop that uses one sentence-concept chunk to link a new sentence to a new concept. That's in the library, not the interpreter.
      - aswanson 19y ago
        That's true, but it's not at all what he said What he implied from culture, I took to mean a polymorphic mapping of "sentences" or "units of communication" in a given understood context that can be learned only through experience. it doesn't stop humans from picking up second languages or from understanding older texts in their native languages It doesn't stop them, but it is far from trivial to learn a new language or to even resolve syntactic,lexical, or sematic ambiguity without context, even in a native speakers language: http://en.wikipedia.org/wiki/Ambiguity http://en.wikipedia.org/wiki/Ambiguity. The ill-posed nature of this mapping is not static, and for people not speaking their native language or young children this becomes the basis for some forms of comedy. As far as I know we have no generic grammar that can generate the rules for all languages; there is no universal "interpreter", and it is not certain that "concept" can be decoupled from "phrase" or "sentence" without context, so I don't think your separation of "library" and "interpreter" holds.
osipov 19y ago
>The meaning of a sentence is formed both by the grammar, which is nearly combinatorial, and by the meanings and connotations of the words used, which are hardly combinatorial at all. I doubt that this sort of thing will just fall out from bayesian networks, or from any simple development based on bayesian networks. The most effective existing approaches to natural language understanding are purely statistics based. This works fairly well (about 85% error rate) in narrow domains. Most approaches aren't even bayesian network based, but use Markov chains/models which can be called a precursor to bayesian networks.
- andreyf 19y ago
  The most effective existing approaches to natural language understanding are purely statistics based. This is where practicality and idealism ought to diverge. Sure, with enough data, we might be able to find spelling mistakes or figure out what pictures someone is looking for. But the techniques we are looking for don't address understanding, (granted they never set out to). Most interesting would be to define meaning and understanding, using strict mathematical models. As McCarty said, language is just the froth on those waves.

dhs 19y ago

>Do you think AI progress is going to continue accelerating for a while? From where I stand, it looks like the more recent progress in AI is due to a shift in research hypotheses. It used to be like "AI is the attempted simulation, or sometimes attempted emulation, of human cognition"; now it's more like "AI is a medium for various forms of human expression". Ms. Koller's work emphasizes numerical expression; the work in Game AI emphasizes visual-artistic and/or logical expression, the work in Natural Language Processing emphasizes expression in natural language. Few people try for machines which pass the Turing Test; many try for machines which capture a small slice of real-life semantics and run with it. As Alan Perlis once wrote: "When we write programs that 'learn', it turns out we do and they don't." I believe that we increasingly realize what he meant with this. And that's good, because not only does it say something about what we cannot do, but it also says what we can. People are increasingly aware of this, go pragmatic (Google/MapReduce proved that you can go quite a ways using brute force), and make progress.

lacker 19y ago

Yes, we will keep making progress in AI. There's two parts to the success of any machine learning program (like the stuff Daphne Koller is working on mentioned in the article). Your algorithm, and your data. To paraphrase Peter Norvig: "Ten times more data will beat any algorithmic improvement." http://glinden.blogspot.com/2006/10/advantages-of-big-data-and-big.html http://glinden.blogspot.com/2006/10/advantages-of-big-data-a... With the rise of the internet, more and more data will be online. Even if we can't figure out smarter ways of analyzing this data, we will at least keep getting more of it. Here's a thought experiment. What if you had a really large dataset of chat logs, so large that any sequence of 1000 words, you could find it somewhere in the logs. Then you just wrote a chat bot that 1. Looked up the history of the chat in its logs 2. Said whatever the person in the logs said next 3. Once you got to 1000 words it said "Gotta go, talk to you later!" Seems like that could pass the Turing test to me. Of course since there are about a million English words you would need on the order of 1000000 ^ 1000 = a googol^60 items in your data set. ;-)