3 ms·

It’s a good philosophical exercise to imagine what’s left of the meaning of a word if you strip all context from it. What is a word if it doesn’t reference any

It’s a good philosophical exercise to imagine what’s left of the meaning of a word if you strip all context from it. What is a word if it doesn’t reference anything, if it has no context? Isn’t this like a pointer without referencing an address? I’ll give you that words can reference tokens of other modalities like hearing and such, but that is still the same problem.

nerdponx 4y ago
I'm thinking in the other direction. Of course context reveals (or as in your philosophical exercise, defines) some of the meaning of the word. But how much of the meaning is context-dependent depends on the word and the context itself where it's being used. For example, consider a word that means two different things in two different contexts that would otherwise be very far apart in whatever latent "meaning space" they might exist in. A naive model might end up averaging across both contexts, resulting in a representation of that word that represents neither meaning of the word, being located vaguely in between both regions of space. Alternatively, this could pull "groups" of words that otherwise are not related into adjacent regions of space, creating the appearance of semantic similarity when there shouldn't be any. In this example, the physical surrounding word context is actually misleading, and addition information is needed in order to make the learned embeddings more realistic. In any case, it should be obvious to people trained in statistical practice that word models do not encode "meaning" per se, but rather an estimate of a model of meaning, which is not a complete model, but it does a pretty good job most of the time. We as the experts should be careful to stick to language along these lines, so as not to promulgate fanciful ideas about AI. I am not even going to bother with the "it's not really intelligent!" angle here. For all I care, the model is fully sentient. But the fact is that the token embeddings do not encode "meaning", it encodes "an estimate of model of meaning". They are our best effort attempt at encoding meaning parsimoniously, but that is not the same thing as meaning itself. Map is not the territory, all models are wrong, etc.
- SanderNL 4y ago
  > Map is not the territory I think this one will get fuzzy real fast when you talk about cognition itself. Otherwise, thanks for the thoughtful response. Food for thought.

nerdponx 4y ago

I'm thinking in the other direction. Of course context reveals (or as in your philosophical exercise, defines) some of the meaning of the word. But how much of the meaning is context-dependent depends on the word and the context itself where it's being used. For example, consider a word that means two different things in two different contexts that would otherwise be very far apart in whatever latent "meaning space" they might exist in. A naive model might end up averaging across both contexts, resulting in a representation of that word that represents neither meaning of the word, being located vaguely in between both regions of space. Alternatively, this could pull "groups" of words that otherwise are not related into adjacent regions of space, creating the appearance of semantic similarity when there shouldn't be any. In this example, the physical surrounding word context is actually misleading, and addition information is needed in order to make the learned embeddings more realistic. In any case, it should be obvious to people trained in statistical practice that word models do not encode "meaning" per se, but rather an estimate of a model of meaning, which is not a complete model, but it does a pretty good job most of the time. We as the experts should be careful to stick to language along these lines, so as not to promulgate fanciful ideas about AI. I am not even going to bother with the "it's not really intelligent!" angle here. For all I care, the model is fully sentient. But the fact is that the token embeddings do not encode "meaning", it encodes "an estimate of model of meaning". They are our best effort attempt at encoding meaning parsimoniously, but that is not the same thing as meaning itself. Map is not the territory, all models are wrong, etc.

SanderNL 4y ago
> Map is not the territory I think this one will get fuzzy real fast when you talk about cognition itself. Otherwise, thanks for the thoughtful response. Food for thought.