If you're specifically focused on semantic similarity, I would say that attention adds to the dimensionality of the vector space. Distances between tokens can

If you're specifically focused on semantic similarity, I would say that attention adds to the dimensionality of the vector space. Distances between tokens can vary depending on context.