Isn't this the same for regular language? If someone is telling something making grammar mistakes, it's likely that the next word will also contain a mistake,

Isn't this the same for regular language? If someone is telling something making grammar mistakes, it's likely that the next word will also contain a mistake, so the LLM reproduces it. And to be fair, I also think that most humans also have this same sensitivity in chess. I've seen even super GMs needing extra effort to realize something like "oh, this opening is just like the Sicilian defense, but with an extra tempo for white", and then instantly "loading" mentally the top strategies for Sicilian defense, that they weren't considering before.