I suspect that at the base level of knowledge are chunks of concept, let's call them nodes. A node would appear at the weighted center of all the words that are

I suspect that at the base level of knowledge are chunks of concept, let's call them nodes. A node would appear at the weighted center of all the words that are synonyms for that concept. Each word has a direction from that node, and if you use words together, there's a vector between them, a direction. Like the recent thread[1] about autoencoders that could find very low dimensional representations of lava lamps, and other chaotic systems, I think that's how transformers and word2vec operate. Also recall that someone had built a tool[2] to find notes that share n-grams automatically in the background, this is a similar idea. 1 - https://news.ycombinator.com/item?id=32294403 2 - https://news.ycombinator.com/item?id=32282260