3 ms·

> embedding the word "snow" in a sufficiently rich abstraction What do you mean by this? (I could have written my comment more politely. Didn't mean to offend

by rand_r 10y ago

> embedding the word "snow" in a sufficiently rich abstraction What do you mean by this? (I could have written my comment more politely. Didn't mean to offend.)

SomeStupidPoint 10y ago
I'm not saying vector spaces are a rich enough structure, but as an example... When we place the word "snow" into a vectorspace of words (along with the other words), we're saying something about what that word means relative to the other words, with those relationships stemming from the experience of people in the real world. "Snow" is similar to "ice" along these dimensions and is similar to "powder" along those dimensions. Ideally, we want something like "snow = ice - solid + powder". All the possible equations for snow in the embedding gives you, essentially, the different ways in which people understand snow. The utility of these structures for NLP is fundamentally that they embed something of the understanding people have for the real world in their embedding of the words. So NLP structures must capture something of "understanding" if they're genuinely useful for NLP. So I agree with the person I was replying to in that you can't have NLP without understanding, because NLP is fundamentally about understanding, but I think they're implying a higher level of understanding than is actually required for most uses of NLP and more over, that it has to be an independent understanding, rather than a secondary derived one.
nl 10y ago
https://demos.explosion.ai/sense2vec/?word=snow&sense=auto https://demos.explosion.ai/sense2vec/?word=snow&sense=auto This is unsupervised learning of word embeddings that shows the "topics" related to the word "snow" and how closely they are related. It shows pretty reasonable understanding of the world - certainly enough to build useful things with it.

3 ms·

> embedding the word "snow" in a sufficiently rich abstraction What do you mean by this? (I could have written my comment more politely. Didn't mean to offend

by rand_r 10y ago

> embedding the word "snow" in a sufficiently rich abstraction What do you mean by this? (I could have written my comment more politely. Didn't mean to offend.)

SomeStupidPoint 10y ago
I'm not saying vector spaces are a rich enough structure, but as an example... When we place the word "snow" into a vectorspace of words (along with the other words), we're saying something about what that word means relative to the other words, with those relationships stemming from the experience of people in the real world. "Snow" is similar to "ice" along these dimensions and is similar to "powder" along those dimensions. Ideally, we want something like "snow = ice - solid + powder". All the possible equations for snow in the embedding gives you, essentially, the different ways in which people understand snow. The utility of these structures for NLP is fundamentally that they embed something of the understanding people have for the real world in their embedding of the words. So NLP structures must capture something of "understanding" if they're genuinely useful for NLP. So I agree with the person I was replying to in that you can't have NLP without understanding, because NLP is fundamentally about understanding, but I think they're implying a higher level of understanding than is actually required for most uses of NLP and more over, that it has to be an independent understanding, rather than a secondary derived one.
nl 10y ago
https://demos.explosion.ai/sense2vec/?word=snow&sense=auto https://demos.explosion.ai/sense2vec/?word=snow&sense=auto This is unsupervised learning of word embeddings that shows the "topics" related to the word "snow" and how closely they are related. It shows pretty reasonable understanding of the world - certainly enough to build useful things with it.