Good point. Though that affects OpenAI too for new data. I had assumed most of their web content was from Common Crawl, and the older pre-ChatGPT Common Crawl

Good point. Though that affects OpenAI too for new data. I had assumed most of their web content was from Common Crawl, and the older pre-ChatGPT Common Crawl datasets used would still be available. But it looks like Twitter, for one, was not in Common Crawl.