I'd be surprised if it was actually raw training data as that would make for an incredibly inefficient/overfitted model. It's probably still just predicting the

I'd be surprised if it was actually raw training data as that would make for an incredibly inefficient/overfitted model. It's probably still just predicting the next token even as it goes off the rails; the context has just changed from a sensible topic to something...broken.