>Because you already have the thought formed before you started typing. Can you prove that GPT-4 doesn't? Clearly there is a sense in which thinks more than on

>Because you already have the thought formed before you started typing. Can you prove that GPT-4 doesn't? Clearly there is a sense in which thinks more than one word ahead, since as I mentioned above it would not otherwise be able to use 'a' vs 'an' correctly. As far as I am aware, exactly to what extent these models have determined what tokens will be generated before they produce anything is an open question in mechanistic interpratability research. I would be very interested if you knew of some work that answers this question empirically.