6 ms·

I'm amazed that apparently, multiplication with an orthogonal matrix (simply rotating and mirroring the word vecs) is able to align two languages!!

• cs702 10y ago
  It's really not that surprising, because word embeddings like word2vec are linear projections from one-hot vectors (i.e., of dimension equal to dictionary size) to dense lower-dimensional vectors, typically of dimension 300 (because in practice 300 works really well). Note also that word embedding models are typically trained to optimize a loss function whose terms include the dot product of embeddings based on the respective words' location in relation to each other in sliding context windows, such that words that are more similar end up with embeddings that have greater cosine similarity, and vice versa. What's incredible (to me) is how well these embeddings work in practice :-)
  • isoprophlex 10y ago
    Thanks for the additional information :) agreed, it's pretty phenomenal stuff..!