3 ms·
Why would an untranslatable word take on bizarre values? If there are enough training examples of the untranslatable word, why couldn't it actually have a sens
by kevinwang 10y ago
Why would an untranslatable word take on bizarre values? If there are enough training examples of the untranslatable word, why couldn't it actually have a sensible mapping in the vectorspace?
p.s. thanks for the reference to that thesis. I've been wondering about uncertainty in predictions for a side project to predict athletic performance that I'm working on, so this is very timely.
- gwern 10y agoword2vec is trying to predict the co-occurence of words; if there is nothing translatable about it, then it will be effectively random. When you fit a maximum likelihood to random stuff and force it to attribute pattern to the noise, it's not going to come up with any sensible mapping.
- kevinwang 10y agoBut just because it's "untranslatable" from, say, Chinese to English, doesn't mean that we can't see many Chinese-Chinese co occurrences of the word, so this Chinese word would still have a vector of significance relative to other Chinese words, right? That is- if you ran word2vec on a Chinese corpus, this word would be represented adequately in the vectorspace. Wouldn't it then make sense, if you could find a proper mapping from one language's space to the other (I don't know if this part is realistic, but maybe by finding correspondences between already-translated words), that this word would not land in a bizarre spot were it to be expressed as a combination of vectors from the other language?