From my limited understanding of how LLMs work, I believe this behavior is enabled by embeddings. The model maps its ~50,000 token vocabulary into a lower-dimen

From my limited understanding of how LLMs work, I believe this behavior is enabled by embeddings. The model maps its ~50,000 token vocabulary into a lower-dimensional vector space. Each dimension in the vector adds some sort of meaning (or at least association) to the word. I saw an example in a Numberphile video where they were able to take the vector of the word "prince", subtract the vector of the word "man", add the vector of the word "woman", and the resulting vector was closest to the word "princess". So in theory there could be a "gender" dimension and a "position of authority" dimension in that vector (or the model might be making other, stranger connections between words that we don't understand). I think the same thing is happening in your example. The model identifies and produces output that keeps it in the general region in the vector space for the "Shakespearean" and "Bowiean" dimensions while still satisfying other requirements.