4 ms·

I was intrigued by the initial transcript, but disappointed that it was edited (lightly). For the question "Why did Mary cheer?" the screenshot showed that the

I was intrigued by the initial transcript, but disappointed that it was edited (lightly). For the question "Why did Mary cheer?" the screenshot showed that the simple AI literally answered "because IT be HER FIRST TIME WINNING". This is correct, but the author edited it into a correct sentence for us. I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? I mean it is quite an accomplishment, but there is no understanding here. In some natural languages with less inflection or change in word order, for example, you could answer any "why" questions with the regex /($question_string) because (.+?)\./ against some source corpus, and then $2 will contain your answer. It doesn't work with English due to slight changes in word order, but surely it would be too much to state that this regex is capable of basic reading comprehension in languages it does work in.... If I'm allowed to massage the question the way the author massaged the output, look at this fine result: http://ideone.com/84QSHC http://ideone.com/84QSHC (output at bottom) Would you say that 14-line Perl program is capable of basic reading comprehension? I wouldn't!

tariqali34 11y ago
>I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? We test whether a human have "basic reading comprehension" by asking them reading comprehension questions. If they answer the questions successfully, then we assume that they have this "reading comprehension" skill. Therefore, if an AI can answer these questions, then it must also have "reading comprehension". Maybe these "reading comprehension questions" don't actually test reading comprehension, just pattern matching and sentence parsing. In which case, we shouldn't be asking these questions to anyone (human or AI). So what we need are new questions.
- kajecounterhack 11y ago
  We test human "reading comprehension" with the assumption that humans have an underlying world model. This is starkly different from a machine whose only ability is information retrieval -- this program (and more advanced versions e.g Freebase) parse and restructure text to be more retrievable. But there is no comprehension because there is no world model. You wouldn't call parsing a google search query "reading comprehension."
  - tariqali34 11y ago
    But if a computer is able to do the same thing as a human does (answer a simple question with pattern recognition and sentence parsing), then the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does. But it's true that we do have a underlying world model by which we understand passages. And that world model is pretty essential if we are to answer more complicated questions. So since it's pretty clear computers can handle the "informational retrieval" side of reading, we should now be focused on generating a "world model" based on text.
    - kajecounterhack 11y ago
      the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does. Yes, it is trivia what we decide to call it, and computers may someday have "intelligence" or "sentience" without having it in the same way that we do. However I think mechanically structuring data and building a querying tool over that cannot be considered "intelligence" or "comprehension" in the sense we seem to speak of here. since it's pretty clear computers can handle the "informational retrieval" side of reading, we should now be focused on generating a "world model" based on text. Yes, that's what Google's working on with Knowledge Graph. It's quite good, it's integrated with search already.
youngprogrammer 11y ago
> I was intrigued by the initial transcript, but disappointed that it was edited (lightly). For the question "Why did Mary cheer?" the screenshot showed that the simple AI literally answered "because IT be HER FIRST TIME WINNING". This is correct, but the author edited it into a correct sentence for us. For some reason, the library I used to get the present tense of "was" is "be". I had a manually fix for this but accidentally removed it when I was cleaning up the code. Sorry if I disappointed you > I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? I would say it is capable of basic reading comprehension because it is attempts to build relationships between different objects although the relationships are weak. When humans do reading comprehension, we do what my program does which is try to parse the sentence and understand the relationships. But brains are also able to augment a lot more information and thus be more flexible with understanding. > Would you say that 14-line Perl program is capable of basic reading comprehension? I wouldn't! I would say it is not because it does not understanding the relation between objects; it only understands that sentences starting with "why" should be answered with everything after the string "because". Also, in your program, if there are two instances of "because" in the source, your program will only choose the first one.
- kajecounterhack 11y ago
  > Would you say that 14-line Perl program is capable of basic reading comprehension? I wouldn't! I would say it is not because it does not understanding the relation between objects; it only understands that sentences starting with "why" should be answered with everything after the string "because". Also, in your program, if there are two instances of "because" in the source, your program will only choose the first one. This line is arbitrary. These programs aren't "understanding" anything, they are just able to generate different facts. Your "AI" is very input-dependent too, which was his point. I am still bothered by the title, as your program doesn't "comprehend" the text -- only you do that. Your program restructures the text and answers to a very limited querying mechanism; there's no comprehension, only information retrieval. They're not the same. Knowledge Graph and Freebase are comparable (/ much larger) efforts.
- 11y ago

tariqali34 11y ago

>I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? We test whether a human have "basic reading comprehension" by asking them reading comprehension questions. If they answer the questions successfully, then we assume that they have this "reading comprehension" skill. Therefore, if an AI can answer these questions, then it must also have "reading comprehension". Maybe these "reading comprehension questions" don't actually test reading comprehension, just pattern matching and sentence parsing. In which case, we shouldn't be asking these questions to anyone (human or AI). So what we need are new questions.

kajecounterhack 11y ago
We test human "reading comprehension" with the assumption that humans have an underlying world model. This is starkly different from a machine whose only ability is information retrieval -- this program (and more advanced versions e.g Freebase) parse and restructure text to be more retrievable. But there is no comprehension because there is no world model. You wouldn't call parsing a google search query "reading comprehension."
- tariqali34 11y ago
  But if a computer is able to do the same thing as a human does (answer a simple question with pattern recognition and sentence parsing), then the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does. But it's true that we do have a underlying world model by which we understand passages. And that world model is pretty essential if we are to answer more complicated questions. So since it's pretty clear computers can handle the "informational retrieval" side of reading, we should now be focused on generating a "world model" based on text.
  - kajecounterhack 11y ago
    the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does. Yes, it is trivia what we decide to call it, and computers may someday have "intelligence" or "sentience" without having it in the same way that we do. However I think mechanically structuring data and building a querying tool over that cannot be considered "intelligence" or "comprehension" in the sense we seem to speak of here. since it's pretty clear computers can handle the "informational retrieval" side of reading, we should now be focused on generating a "world model" based on text. Yes, that's what Google's working on with Knowledge Graph. It's quite good, it's integrated with search already.

youngprogrammer 11y ago

> I was intrigued by the initial transcript, but disappointed that it was edited (lightly). For the question "Why did Mary cheer?" the screenshot showed that the simple AI literally answered "because IT be HER FIRST TIME WINNING". This is correct, but the author edited it into a correct sentence for us. For some reason, the library I used to get the present tense of "was" is "be". I had a manually fix for this but accidentally removed it when I was cleaning up the code. Sorry if I disappointed you > I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? I would say it is capable of basic reading comprehension because it is attempts to build relationships between different objects although the relationships are weak. When humans do reading comprehension, we do what my program does which is try to parse the sentence and understand the relationships. But brains are also able to augment a lot more information and thus be more flexible with understanding. > Would you say that 14-line Perl program is capable of basic reading comprehension? I wouldn't! I would say it is not because it does not understanding the relation between objects; it only understands that sentences starting with "why" should be answered with everything after the string "because". Also, in your program, if there are two instances of "because" in the source, your program will only choose the first one.

kajecounterhack 11y ago
> Would you say that 14-line Perl program is capable of basic reading comprehension? I wouldn't! I would say it is not because it does not understanding the relation between objects; it only understands that sentences starting with "why" should be answered with everything after the string "because". Also, in your program, if there are two instances of "because" in the source, your program will only choose the first one. This line is arbitrary. These programs aren't "understanding" anything, they are just able to generate different facts. Your "AI" is very input-dependent too, which was his point. I am still bothered by the title, as your program doesn't "comprehend" the text -- only you do that. Your program restructures the text and answers to a very limited querying mechanism; there's no comprehension, only information retrieval. They're not the same. Knowledge Graph and Freebase are comparable (/ much larger) efforts.
11y ago