3 ms·

Here is a recent Facebook research paper that addresses the problem I believe this paper is intended to solve. Towards AI-Complete Question Answering: A Set of

Here is a recent Facebook research paper that addresses the problem I believe this paper is intended to solve. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks http://arxiv.org/pdf/1502.05698v6.pdf http://arxiv.org/pdf/1502.05698v6.pdf This recent paper is an algorithm to solve simple question answering from the context of a conversation... like the ones in the older paper... for instance. John is in the playground. Bob is in the office. Where is John? A:playground I believe a traditional LSTM RNN would have likely cleared the information about John from from it's long short term memory vector. Whereas you would still have access to the information about John in a stack. fta: "Since the machine can essentially learn how to operate its memory, it can learn how to program itself by producing sequences of instructions."

cs702 11y ago
I have shivers going down my spine at the idea of feeding, say, skip-thought vectors representing sentences[1] into stack-augmented recurrent neural nets as proposed in Facebook's paper[2], for all sorts of interesting tasks. -- [1] http://arxiv.org/pdf/1506.06726v1.pdf http://arxiv.org/pdf/1506.06726v1.pdf [2] http://arxiv.org/pdf/1503.01007.pdf http://arxiv.org/pdf/1503.01007.pdf
Smerity 11y ago
If people are interested in playing with the dataset from "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks", I added an example to Keras that uses recurrent neural networks[1]. It achieves similar accuracy to the baseline LSTM approach in the paper. I also did a write-up that explains the task and architecture[2]. The dataset (bAbi tasks[3]) released by Facebook is interesting but has a number of limitations. The code used to produce it is also not open source. If someone has the interest and/or past experience in MUD style environments, creating an open source dataset would be amazing! [1]: https://github.com/fchollet/keras/blob/master/examples/babi_rnn.py https://github.com/fchollet/keras/blob/master/examples/babi_... [2]: http://smerity.com/articles/2015/keras_qa.html http://smerity.com/articles/2015/keras_qa.html [3]: https://research.facebook.com/researchers/1543934539189348 https://research.facebook.com/researchers/1543934539189348
- nl 11y ago
  Oh!! This is something I must try. I remember you've mentioned issues with the bAbi question set previously. Have you (or anyone else) tried using the same approach on a non-artificial dataset? I'm thinking something like blending this with the approach Richard Socher used on the Quizbowl code[1]. [1] http://cs.umd.edu/~miyyer/qblearn/ http://cs.umd.edu/~miyyer/qblearn/
mjn 11y ago
I also like this vaguely related paper from one of the ex-IBM-Watson team members: http://arxiv.org/abs/1508.06924 http://arxiv.org/abs/1508.06924 "Using Thought-Provoking Children's Questions to Drive Artificial Intelligence Research"
nl 11y ago
Weston uses Memory Networks[1] to mostly solve this set of tasks. Some researches out of Huawei and I think Hong Kong University had some recent success using a different technique on the tasks memory networks failed at[2]. [1] http://arxiv.org/abs/1506.02075 http://arxiv.org/abs/1506.02075 [2] http://arxiv.org/abs/1508.05508 http://arxiv.org/abs/1508.05508

cs702 11y ago

I have shivers going down my spine at the idea of feeding, say, skip-thought vectors representing sentences[1] into stack-augmented recurrent neural nets as proposed in Facebook's paper[2], for all sorts of interesting tasks. -- [1] http://arxiv.org/pdf/1506.06726v1.pdf http://arxiv.org/pdf/1506.06726v1.pdf [2] http://arxiv.org/pdf/1503.01007.pdf http://arxiv.org/pdf/1503.01007.pdf

Smerity 11y ago

If people are interested in playing with the dataset from "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks", I added an example to Keras that uses recurrent neural networks[1]. It achieves similar accuracy to the baseline LSTM approach in the paper. I also did a write-up that explains the task and architecture[2]. The dataset (bAbi tasks[3]) released by Facebook is interesting but has a number of limitations. The code used to produce it is also not open source. If someone has the interest and/or past experience in MUD style environments, creating an open source dataset would be amazing! [1]: https://github.com/fchollet/keras/blob/master/examples/babi_rnn.py https://github.com/fchollet/keras/blob/master/examples/babi_... [2]: http://smerity.com/articles/2015/keras_qa.html http://smerity.com/articles/2015/keras_qa.html [3]: https://research.facebook.com/researchers/1543934539189348 https://research.facebook.com/researchers/1543934539189348

nl 11y ago
Oh!! This is something I must try. I remember you've mentioned issues with the bAbi question set previously. Have you (or anyone else) tried using the same approach on a non-artificial dataset? I'm thinking something like blending this with the approach Richard Socher used on the Quizbowl code[1]. [1] http://cs.umd.edu/~miyyer/qblearn/ http://cs.umd.edu/~miyyer/qblearn/

mjn 11y ago

I also like this vaguely related paper from one of the ex-IBM-Watson team members: http://arxiv.org/abs/1508.06924 http://arxiv.org/abs/1508.06924 "Using Thought-Provoking Children's Questions to Drive Artificial Intelligence Research"

nl 11y ago

Weston uses Memory Networks[1] to mostly solve this set of tasks. Some researches out of Huawei and I think Hong Kong University had some recent success using a different technique on the tasks memory networks failed at[2]. [1] http://arxiv.org/abs/1506.02075 http://arxiv.org/abs/1506.02075 [2] http://arxiv.org/abs/1508.05508 http://arxiv.org/abs/1508.05508