Training on self generated data is not necessarily such a problem, see how successful alphazero/muzero is when it is only trained on self play. The key is that

Training on self generated data is not necessarily such a problem, see how successful alphazero/muzero is when it is only trained on self play. The key is that you need some kind of external indicator that tells you which generated examples are good and which are bad. In the case of alphazero you get that by simulating games and seeing who wins, in the case of LLMs you will be only taking the generations that are 'successful', e.g. which HN posts get upvoted.