Everybody training models on large amounts of lightly filtered internet text is partially distilling every other model that had its output posted verbatim to th

Everybody training models on large amounts of lightly filtered internet text is partially distilling every other model that had its output posted verbatim to the internet.