1. You are just referring to fine tuning, I am referring to training the base model. 2. They still saw performance improvements which is why they did train on

1. You are just referring to fine tuning, I am referring to training the base model. 2. They still saw performance improvements which is why they did train on the data multiple times, you can see in the paper. 3. there was a recent paper demonstrating that reusing data still saw continued improvements in perplexity, i am on my ipad so cannot find it now