If you define the state broadly enough to include the state of the entire machine itself, including all of its internal representations, weights, activations, e

If you define the state broadly enough to include the state of the entire machine itself, including all of its internal representations, weights, activations, etc, then you are playing a funny trick here. By the same reasoning, a human brain is also Markov process. What you are doing here is a vast oversimplification and it is practically useless for understanding how LLMs work.