3 ms·

The article munges thought vectors and word vectors, but it really is quite startling what's possible. Take a word vector for "Paris", add the vector for "Germ

by panarky 11y ago

The article munges thought vectors and word vectors, but it really is quite startling what's possible. Take a word vector for "Paris", add the vector for "Germany", subtract the vector for "France", and the result is "Berlin". indexes, metrics = model.analogy(pos=['paris', 'germany'], neg=['france'], n=10) (u'berlin', 0.32333651414395953, 20) Source: http://nbviewer.ipython.org/github/danielfrg/word2vec/blob/master/examples/word2vec.ipynb http://nbviewer.ipython.org/github/danielfrg/word2vec/blob/m...

siegecraft 11y ago
Is that really that surprising, though? Assuming you train it on encyclopedias and reference material, you'd get a ton of "X is the capital of Y" sentences.
- danieldk 11y ago
  In my experience it also works fairly well when training e.g. on newspaper text. It's not really that surprising when you think about how it works. Similar words cluster together in vector space in some dimensions. E.g. 'Paris' and 'Berlin' will both have capital-ish contexts. However, they also are different in some ways, e.g. Paris will have France-ish contexts and Berlin German-ish contexts. The 2-dimensional PCA projection in figure 2 of one of Mikolov's papers [1] gives an intuition why substraction/addition generally works. [1] http://arxiv.org/pdf/1310.4546.pdf http://arxiv.org/pdf/1310.4546.pdf
calebm 11y ago
That is awesome.
duaneb 11y ago
I mean, not that startling if you grasp the underlying modeling. Thought vectoring would obviously be much harder because contextual awareness would be necessary.
iandanforth 11y ago
If you like this kind of stuff check out cortical.io
pella 11y ago
Similar in "semantic fingerprinting": * Jaguar – Porsche = Tiger * Apple – Fruit = Computer more: * http://www.cortical.io/technology_semantic.html http://www.cortical.io/technology_semantic.html * http://numenta.org/ http://numenta.org/ NuPIC is an open source project based on a theory of neocortex called Hierarchical Temporal Memory (HTM)

by panarky 11y ago

siegecraft 11y ago

Is that really that surprising, though? Assuming you train it on encyclopedias and reference material, you'd get a ton of "X is the capital of Y" sentences.

danieldk 11y ago
In my experience it also works fairly well when training e.g. on newspaper text. It's not really that surprising when you think about how it works. Similar words cluster together in vector space in some dimensions. E.g. 'Paris' and 'Berlin' will both have capital-ish contexts. However, they also are different in some ways, e.g. Paris will have France-ish contexts and Berlin German-ish contexts. The 2-dimensional PCA projection in figure 2 of one of Mikolov's papers [1] gives an intuition why substraction/addition generally works. [1] http://arxiv.org/pdf/1310.4546.pdf http://arxiv.org/pdf/1310.4546.pdf

calebm 11y ago

That is awesome.

duaneb 11y ago

I mean, not that startling if you grasp the underlying modeling. Thought vectoring would obviously be much harder because contextual awareness would be necessary.

iandanforth 11y ago

If you like this kind of stuff check out cortical.io

pella 11y ago

Similar in "semantic fingerprinting": * Jaguar – Porsche = Tiger * Apple – Fruit = Computer more: * http://www.cortical.io/technology_semantic.html http://www.cortical.io/technology_semantic.html * http://numenta.org/ http://numenta.org/ NuPIC is an open source project based on a theory of neocortex called Hierarchical Temporal Memory (HTM)