I don't think anyone's saying they're Markov chains with context length 1. In all the critiques I've read no one's even come close to articulating that. But th

I don't think anyone's saying they're Markov chains with context length 1. In all the critiques I've read no one's even come close to articulating that. But they are Markov chains with context length N. They're approximating (implicit) Markov transition matrices through a fancier method of computation but they still are just generating one token at a time.