The solution is to weigh the data by its engagement. If in a hypothetical scenario the internet is littered with 90% Ai generated trash, then human users will o

The solution is to weigh the data by its engagement. If in a hypothetical scenario the internet is littered with 90% Ai generated trash, then human users will only be engaging with the 10% of valuable content, which may have been created by humans or AI. By training only on this data (or, more smoothly, somehow weighing your data by its human engagement), you have a reinforcement learning signal from an average of intelligent internet users that can steer you towards a good/better model.