3 ms·

Is this really reading comprehension, being able to fill in a template by finding facts in the story? I mean, I get that it's able to parse the query and find t

Is this really reading comprehension, being able to fill in a template by finding facts in the story? I mean, I get that it's able to parse the query and find the best fact within the story to fit that slot, but this is more like a better search indexer/Google Factbook then true comprehension. My daughter is in 3rd grade, and 3rd grade reading comprehension requires an ability to make inferences about the text and reasons behind people's acts even if the verbatim text isn't present. How could a pure deep learning approach work here, when to be able to make inferences, you'd need knowledge about an enormous number of entities. Would this systems be able to answer a question like "Where did Snow White's friends or housemates go" if the word "friend" or "housemate" weren't in the text? Or perhaps an easy question for a child: "How do you think Snow White felt after the dwarves left?"

qwantum30 10y ago
So according to wikipedia reading comprehension is defined as the ability to read text, process it, and understand its meaning. So a fill in the blanks question is in fact reading comprehension if the query is not in the text itself which is the case in the dataset. Text search is really the opposite where you are trying to find words that are related to the query and a better google would be to try to find words that are semantically related to the query. You could look at this as inverse text search as given keywords find the related word. What you described as comprehension for your daughter is also an example of comprehension but it is not the only one. Yes it is more advanced than what is in these datasets but the field is only at its initial stages and there is more to come in the future. That being said I don't see why a neural network could not infer that dwarfs and snow white are related to the word friends given that one can feed neural networks both general text and story text. In the same way that a child would have to infer from reading general text and story text that snow white and the dwarves were friends.
andrewljohnson 10y ago
Would this systems be able to answer a question like "Where did Snow White's friends or housemates go" if the word "friend" or "housemate" weren't in the text? A compounded system would. This fill-in-the-blank neural net, combined with another neural net to do synonyms (match the word from text that would align with friend/housemate)... that's like Word Vectors: https://www.tensorflow.org/versions/r0.9/tutorials/word2vec/index.html https://www.tensorflow.org/versions/r0.9/tutorials/word2vec/.... I think your daughter has these sorts of learnings and more. Also, if we give a computer/robot a bunch of these neural nets (reading, word-veccing, etc), and it is also programmed as a chatbot, we're going to start having a lot more existential questions about consciousness. This convergence of simple systems to form "consciousness" seems to match some recent evolutionary science too:(http://www.theatlantic.com/science/archive/2016/06/how-consciousness-evolved/485558/ http://www.theatlantic.com/science/archive/2016/06/how-consc.... Though maybe I'm not making a fair leap.

qwantum30 10y ago

So according to wikipedia reading comprehension is defined as the ability to read text, process it, and understand its meaning. So a fill in the blanks question is in fact reading comprehension if the query is not in the text itself which is the case in the dataset. Text search is really the opposite where you are trying to find words that are related to the query and a better google would be to try to find words that are semantically related to the query. You could look at this as inverse text search as given keywords find the related word. What you described as comprehension for your daughter is also an example of comprehension but it is not the only one. Yes it is more advanced than what is in these datasets but the field is only at its initial stages and there is more to come in the future. That being said I don't see why a neural network could not infer that dwarfs and snow white are related to the word friends given that one can feed neural networks both general text and story text. In the same way that a child would have to infer from reading general text and story text that snow white and the dwarves were friends.

andrewljohnson 10y ago

Would this systems be able to answer a question like "Where did Snow White's friends or housemates go" if the word "friend" or "housemate" weren't in the text? A compounded system would. This fill-in-the-blank neural net, combined with another neural net to do synonyms (match the word from text that would align with friend/housemate)... that's like Word Vectors: https://www.tensorflow.org/versions/r0.9/tutorials/word2vec/index.html https://www.tensorflow.org/versions/r0.9/tutorials/word2vec/.... I think your daughter has these sorts of learnings and more. Also, if we give a computer/robot a bunch of these neural nets (reading, word-veccing, etc), and it is also programmed as a chatbot, we're going to start having a lot more existential questions about consciousness. This convergence of simple systems to form "consciousness" seems to match some recent evolutionary science too:(http://www.theatlantic.com/science/archive/2016/06/how-consciousness-evolved/485558/ http://www.theatlantic.com/science/archive/2016/06/how-consc.... Though maybe I'm not making a fair leap.