This is a nice bridge between embeddings common in AI and the symbolic reasoning that we have a lot of theoretical understanding of. However, I'm not sure what

This is a nice bridge between embeddings common in AI and the symbolic reasoning that we have a lot of theoretical understanding of. However, I'm not sure what the discovery is. Multiplication, addition, and recovering the bases of vectors are high-school / college-freshman-level linear algebra concepts. Can anyone shed more light on the 'discovery'? It seems like an interesting idea, but I'm surprised that Quanta (who typically has very high quality articles) is making a big deal about vector operations that are well understood. In particular, the use of addition to create a superposition of embeddings is many decades old (it's the basis of the bag of words approach). Multiplication as the 'description' operator is perhaps interesting. Using change of basis to tease out the coefficients for linear combinations of vectors is... basic linalg. All in all, it seems the main 'discovery' here is a way to generate richer training vectors than the typical 'one-hot' approach that we use today. Although, given that one-hot vectors are also orthogonal, it's not obvious to me why a random orthogonal vector is better than this approach. Perhaps they're going for dimensionality reduction, but if you have ten classes, then you need ten vectors and you need a 10-D output vector. You can't get 10 orthogonal vectors in 9-dimensional space. Given that there is no perf difference in computing with orthogonal one-hot vectors and orthogonal random vectors, I'm again puzzled by the 'discovery'. In fact, one hot vectors are slightly cheaper to compute with since they have zeros.