7 ms·

They advertise it as "speech-to-meaning". I've been thinking about meaning in AI and how important it is for understanding and interacting with the world to be

by tomp 11y ago

They advertise it as "speech-to-meaning". I've been thinking about meaning in AI and how important it is for understanding and interacting with the world to be able to answer "What does this mean?" and "Does this make sense?" Does anyone hove any insight or references to recent research about how to model, train for and represent meaning?

jdiez17 11y ago
Check this[1] out: it describes some techniques used to convert natural language to machine-readable descriptions. [1] https://github.com/wiseman/energid_nlp https://github.com/wiseman/energid_nlp
nl 11y ago
Here's a chronological reading list of techniques Traditional NLP -> 2012, IBM Watson papers[1] Watson is probably the pinnacle of "traditional" style NLP (ie, Tokenization/Lemmatization/Part of Speech Tagging/Framing/Knowledge Engineering/etc) Word2Vec -> 2013, Google paper[2] Word2Vec kind of exploded everyone's mind for a while. Subgraph Embedding -> 2015, Facebook AI Group [3][4] [1] http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717 http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=617771... [2] http://arxiv.org/pdf/1301.3781.pdf http://arxiv.org/pdf/1301.3781.pdf [3] https://research.facebook.com/publications/1473550739586509/question-answering-with-subgraph-embeddings/ https://research.facebook.com/publications/1473550739586509/... [4] http://arxiv.org/abs/1502.05698 http://arxiv.org/abs/1502.05698
- syllogism 11y ago
  You might be interested to read this: http://arxiv.org/pdf/1402.3722v1.pdf http://arxiv.org/pdf/1402.3722v1.pdf word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method. Yoav Goldberg and Omer Levy. arXiv 2014. [pdf] The word2vec software of Tomas Mikolov and colleagues has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to ﬁgure out the rationale behind the equations.
  - nl 11y ago
    Yeah. Pulling a quote: Why does this produce good word representations? Good question. We don’t really know. The objective above clearly tries to increase the quantity vw·vc for good word-context pairs, and decrease it for bad ones. Intuitively, this means that words that share many contexts will be similar to each other (note also that contexts sharing many words will also be similar to each other). This is, however, very hand-wavy. I love an honest paper.
syllogism 11y ago
On limited network connection, so can't give you links, but: * Academic keyword is "semantic parsing", main publications open access at http://aclweb.org/anthology http://aclweb.org/anthology * Semantic parsing maps sentences into something actionable (a query is answered, the robot moves, etc). For a more abstract, non-task specific version, relevant keywords are "semantic role labelling", and recently "the AMR corpus". Systems stuff falls under "entailment" and "paraphrase". * The theory driving this stuff is increasingly combinatory categorial grammar (CCG). Read Mark Steedman's book "The Syntactic Process", for an explanation of how to parse NL sentences into lambda terms. The premise of the theory is that we can make an efficient compiler that outputs the lambda terms directly, and that the syntactic structure is just the derivation structure --- a trace of the algorithm. At the end of the day though, we just want the output. There are lots of ways to compute the relevant reepresentations. * Much relevant work on CCG-based semantic parsing is happening at University of Washington, e.g. under Luke Zettlemoyer. With the new Allen Institute investing serious money, Seattle looks insanely good for NLP now.

by tomp 11y ago

jdiez17 11y ago

Check this[1] out: it describes some techniques used to convert natural language to machine-readable descriptions. [1] https://github.com/wiseman/energid_nlp https://github.com/wiseman/energid_nlp

nl 11y ago

Here's a chronological reading list of techniques Traditional NLP -> 2012, IBM Watson papers[1] Watson is probably the pinnacle of "traditional" style NLP (ie, Tokenization/Lemmatization/Part of Speech Tagging/Framing/Knowledge Engineering/etc) Word2Vec -> 2013, Google paper[2] Word2Vec kind of exploded everyone's mind for a while. Subgraph Embedding -> 2015, Facebook AI Group [3][4] [1] http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=6177717 http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=617771... [2] http://arxiv.org/pdf/1301.3781.pdf http://arxiv.org/pdf/1301.3781.pdf [3] https://research.facebook.com/publications/1473550739586509/question-answering-with-subgraph-embeddings/ https://research.facebook.com/publications/1473550739586509/... [4] http://arxiv.org/abs/1502.05698 http://arxiv.org/abs/1502.05698

syllogism 11y ago
You might be interested to read this: http://arxiv.org/pdf/1402.3722v1.pdf http://arxiv.org/pdf/1402.3722v1.pdf word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method. Yoav Goldberg and Omer Levy. arXiv 2014. [pdf] The word2vec software of Tomas Mikolov and colleagues has gained a lot of traction lately, and provides state-of-the-art word embeddings. The learning models behind the software are described in two research papers. We found the description of the models in these papers to be somewhat cryptic and hard to follow. While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to ﬁgure out the rationale behind the equations.
- nl 11y ago
  Yeah. Pulling a quote: Why does this produce good word representations? Good question. We don’t really know. The objective above clearly tries to increase the quantity vw·vc for good word-context pairs, and decrease it for bad ones. Intuitively, this means that words that share many contexts will be similar to each other (note also that contexts sharing many words will also be similar to each other). This is, however, very hand-wavy. I love an honest paper.

syllogism 11y ago

On limited network connection, so can't give you links, but: * Academic keyword is "semantic parsing", main publications open access at http://aclweb.org/anthology http://aclweb.org/anthology * Semantic parsing maps sentences into something actionable (a query is answered, the robot moves, etc). For a more abstract, non-task specific version, relevant keywords are "semantic role labelling", and recently "the AMR corpus". Systems stuff falls under "entailment" and "paraphrase". * The theory driving this stuff is increasingly combinatory categorial grammar (CCG). Read Mark Steedman's book "The Syntactic Process", for an explanation of how to parse NL sentences into lambda terms. The premise of the theory is that we can make an efficient compiler that outputs the lambda terms directly, and that the syntactic structure is just the derivation structure --- a trace of the algorithm. At the end of the day though, we just want the output. There are lots of ways to compute the relevant reepresentations. * Much relevant work on CCG-based semantic parsing is happening at University of Washington, e.g. under Luke Zettlemoyer. With the new Allen Institute investing serious money, Seattle looks insanely good for NLP now.