4 ms·
Word vectors for the same concept in different languages are very close, that's not really surprising. The whole `v_king - v_male + v_female = v_queen` idea lea
by pbhjpbhj 2y ago
Word vectors for the same concept in different languages are very close, that's not really surprising. The whole `v_king - v_male + v_female = v_queen` idea leads to an intuition that `v_grazie - v_italien + v_deutsch = v_danke`.
Code switching probably appears a lot in both literature and audio from some contexts too.
Why would it matter that a concept is expressed in a different human language? How would you structure a multilingual LLM that it couldn't syncretise languages in this way? You'd seemingly need to properly segregate the languages.
- fcatalan 2y agoYes I understand the encodings will be close and that helps, I guess that's why they produce good translations, but I'm intrigued by the LLM having so much control of the switching without even explicit prompting, just a one-shot example. I also guess I'm easily impressed.
- nine_k 2y agoI suppose this mechanism, only quite a bit up the conceptual ladder, is necessary for LLMs to be able to translate between languages, which they apparently are trained to do, explicitly or not.