It shouldn't be limited to censoring. Fine tuning is done on small set of some domain specific samples. Surely that results in gradual 'forgetting' of the alrea

It shouldn't be limited to censoring. Fine tuning is done on small set of some domain specific samples. Surely that results in gradual 'forgetting' of the already learned data from big set. If run for long time model will relearn on the tuning set, and overfit most likely.