a markov chain can have an internal representation related to the intention of the user. I guess this example just got copied a lot and is therefore included mu

a markov chain can have an internal representation related to the intention of the user. I guess this example just got copied a lot and is therefore included multiple times in the training data, forcing the network to memorise it. Neural networks always memorise things that appear too frequent. Memorized Artifacts in an otherwise working neural network is usually seen as a "bug" (since the training allowed the network to cheat), not as a proof that the network didn't generalise.