3 ms·
Could you give an example of an embedding space that is only meaningful locally? The word vector idea of modifying a word by some operation e.g King->Queen ~=
by extasia 3y ago
Could you give an example of an embedding space that is only meaningful locally?
The word vector idea of modifying a word by some operation e.g King->Queen ~= man->woman ~= uncle->aunt
Which intuitively seems non-local to me.
- jerf 3y agoToss that intuition out. That's in some sense the major problem with higher dimensional spaces; our intuition about locality is built in a super-small-dimensional space. Compare the locality of King, man, and uncle to King, lemma, and titrate. The former set is quite local.
- tysam_and 3y agoThe parent commenter in this case is referencing the tendency for high dimensional spaces to very quickly go "off the rails" in that, if I understand correctly, most smooth spaces are locally flat, but as you get further and further away, the curvature can really screw with things. Even with that fixed, Euclidean and otherwise measures do not translate intuitively to high dimensions outside of much more local spaces.