3 ms·

It seems like an old way to solve the problem. I would be quite surprised if it gave good results compared to recent advances such as the LSTM-based paper “A Ne

It seems like an old way to solve the problem. I would be quite surprised if it gave good results compared to recent advances such as the LSTM-based paper “A Neural Conversational Model” [0]. In fact, I doubt it is even competitive compared to Cleverbot. It does try to compare itself with existing alternatives, but it does not characterize the pros and cons of each alternative and does not compare results. (I am not an expert, but did take classes on the subject of pattern recognition and machine learning.) [0]: http://arxiv.org/pdf/1506.05869v1.pdf http://arxiv.org/pdf/1506.05869v1.pdf

creamyhorror 11y ago
For anyone getting up to date with language AI, taliesinb recommended this paper as "a real eye-opener" in the discussion about Google's Go AI https://news.ycombinator.com/item?id=10984973 https://news.ycombinator.com/item?id=10984973 : "A Cognitive Neural Architecture Able to Learn and Communicate through Natural Language" http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0140866 http://journals.plos.org/plosone/article?id=10.1371%2Fjourna... It uses a more complex (I daresay more advanced) model than the LSTM paper you linked, a model that can be trained to learn and use language constructs (in his words, "mimic certain kinds of child-level cognition with TINY amounts of training data"). Definitely worth a read. This ANNABELL model doesn't require much training input to go from blank slate to conversation-capable (1587 input sentences -> 521 output sentences; see Appendix S1 for examples). This high learning efficiency might imply that it indeed resembles language-processing architecture in humans (which it was modelled after). At the low level it's also neurons, but organised and connected into specific, planned subsystems. A key point is that the central executive (i.e. the core, which controls the flow of data between slave systems) is a trainable neural network itself, which learns to generate the "mental actions" that control the flow of data between slave systems (like short-term and long-term memory), rather than rely on fixed rules to control the flow. This apparently allows the system to generalise.
- mark_l_watson 11y ago
  +1 Thanks for the link, I just added this to my eval list.
- nacnud 11y ago
  Fascinating reading. This is probably of interest to the wider HN crowd.

creamyhorror 11y ago

For anyone getting up to date with language AI, taliesinb recommended this paper as "a real eye-opener" in the discussion about Google's Go AI https://news.ycombinator.com/item?id=10984973 https://news.ycombinator.com/item?id=10984973 : "A Cognitive Neural Architecture Able to Learn and Communicate through Natural Language" http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0140866 http://journals.plos.org/plosone/article?id=10.1371%2Fjourna... It uses a more complex (I daresay more advanced) model than the LSTM paper you linked, a model that can be trained to learn and use language constructs (in his words, "mimic certain kinds of child-level cognition with TINY amounts of training data"). Definitely worth a read. This ANNABELL model doesn't require much training input to go from blank slate to conversation-capable (1587 input sentences -> 521 output sentences; see Appendix S1 for examples). This high learning efficiency might imply that it indeed resembles language-processing architecture in humans (which it was modelled after). At the low level it's also neurons, but organised and connected into specific, planned subsystems. A key point is that the central executive (i.e. the core, which controls the flow of data between slave systems) is a trainable neural network itself, which learns to generate the "mental actions" that control the flow of data between slave systems (like short-term and long-term memory), rather than rely on fixed rules to control the flow. This apparently allows the system to generalise.

mark_l_watson 11y ago
+1 Thanks for the link, I just added this to my eval list.
nacnud 11y ago
Fascinating reading. This is probably of interest to the wider HN crowd.