My guess: because it's useful with following analogy. We use text embeddings to represent concepts written words. They lack the nuance, when the same word has

My guess: because it's useful with following analogy. We use text embeddings to represent concepts written words. They lack the nuance, when the same word has different meaning in different concepts. LLM's use text embeddings and enrich it with the Attention mechanism. For words, that in reality are used to represent a single concept, a text embedding is working perfectly on it's own. For those concepts that are context dependent, we're using the attention mechanism to gently guide the text embedding closer to the intended, identified by surrounding words, meaning. That's the role of the Value vector of the K, Q, V triplet in the Attention mechanism to be precise. So, this is a simplistic approach, which corresponds to the "first approximation", which could be good enough for some cases. We don't know which exactly yet, but we'll know, once enough evidence is given to the contrary. It's not a good model, but a very good approach in order to do research in a stepwise manner. With time, it'll get more and more nuanced, one approximation at a time.