I believe the result might be similar to when one does oversampling to avoid class imbalances. When doing oversampling it happens that exact duplicates are trai

I believe the result might be similar to when one does oversampling to avoid class imbalances. When doing oversampling it happens that exact duplicates are trained. This is equivalent to increasing the weight for these examples. This might fix the class imbalance but at the cost of increasing the bias of the model. This means the model might overfit to the duplicates shown in the training set. In the case of LLM trained on mostly LLM generated data you will see a similar increase in bias of the model and a overfitting to LLM generated data. This might lead to limitations of the performance of LLM in the future.