It's perfectly fine, theoretically for the Markov chain to depend on more than the previous token. When you start to do that though, unless you can write down a

It's perfectly fine, theoretically for the Markov chain to depend on more than the previous token. When you start to do that though, unless you can write down a function (or approximation, like an NN) the number of states in your state space that you have to enumerate explodes very quickly. This is the simple motivation behind stuff like GPT or neural reinforcement learning.