3 ms·
An Embedding is a n-dimensional vector (think of it as a sequence of n numbers). During training, each token (or word) gets an Embedding assigned. Critically,
by pallas_athena 4y ago
An Embedding is a n-dimensional vector (think of it as a sequence of n numbers).
During training, each token (or word) gets an Embedding assigned.
Critically, _similar words will get similar embeddings_. And "similar words" could mean both semantically or (as was the example) syntactically ("apple" and "appli").
And being vectors, you can do operations on them. To give the classic example, you could do:
Embedding(`king`) + Embedding(`female`) = Embedding(`queen`).
- pertymcpert 4y agoIt's completely mind blowing that you can add the vectors like that.