As part of the training loop I will periodically ask the model a question. At first the model will respond normally, and then get progressively more repetitive

As part of the training loop I will periodically ask the model a question. At first the model will respond normally, and then get progressively more repetitive until it starts repeating tokens from the training data. The point in time where the model starts repeating itself aligns with the sudden change in the visualization.