4 ms·
Now it's `one_hot_focus x W1 x (one_hot_context x W2)^T`. So we still pick one row of the matrix from the focus and context embeddings, but they're separate emb
by bollu 7y ago
Now it's `one_hot_focus x W1 x (one_hot_context x W2)^T`. So we still pick one row of the matrix from the focus and context embeddings, but they're separate embeddings.
- MichaelStaniek 7y agoYes, but thats also what happens in the normal formulation, no? So the second weight matrix actually are our context embeddings?
- avn2109 7y agoNew user "MichaelStaniek" has a grammatical, relevant, good-faith sibling comment to this which is (imho) inexplicably banned, almost certainly due to a mistake, and I hope somebody will unban him.