6 ms·
This is my understanding as well. There a somewhat accessible introductory video I found useful [0]. The algebra makes it possible to encode sets, key/value as
by chombier 3y ago
This is my understanding as well. There a somewhat accessible introductory video I found useful [0].
The algebra makes it possible to encode sets, key/value associations and sequences to build a knowledge base, and the dot product provides a similarity measure for querying the base.
IIUC the key is that for large space dimensions, any two random vectors (say with uniform distribution over {-1, +1}^d) are almost guaranteed to be near- orthogonal.
This makes it easy to add new items to the base (by sampling a new random vector and updating the base using algebraic operations with other items), yet the amount of noise introduced by near-orthogonality remains controlled and can be filtered out to keep the algebraic structure working as the base grows.
Honestly it seems a bit too good to be true, I'd be very interested to see what are the tradeoffs in practice.
[0] https://www.youtube.com/watch?v=oB_mHCurNCI https://www.youtube.com/watch?v=oB_mHCurNCI