This description falls apart for two reasons 1. It only accurately describes pre-training 2. It ignores the existence of generalization Next token prediction

This description falls apart for two reasons 1. It only accurately describes pre-training 2. It ignores the existence of generalization Next token prediction is just a training task, not "what the model does internally" in any meaningful sense