I think you misunderstand machine learning. "probability of each token appearing given the previous tokens" is how humans write code too: We write code based on

I think you misunderstand machine learning. "probability of each token appearing given the previous tokens" is how humans write code too: We write code based on what we want to do and what we have written before. "what I want to do" was captured in the comments added.