Depends what you want and whether you have new data or just train for more epochs. More relevant data is probably always a good thing, repeating the same data n

Depends what you want and whether you have new data or just train for more epochs. More relevant data is probably always a good thing, repeating the same data not necessarily. Repeating data is probably good though if you really want the model to learn certain things. State-of-the-art training curriculums probably train on low-quality data to teach the model how to handle language. You don't want to repeat that one too often, you rather want a lot of diverse input. Data augmentation should help here too, for example deliberately introducing spelling errors in the input and expecting the model to return the unchanged input. Later stages contain high-quality data and try to improve performance on specific tasks. That probably benefits from repetition. The final stage is alignment to chat or instruct prompts and safety engineering.