5 ms·

Officially OpenAI uses CommonCrawl, or at least used to do so.

• morkalork 4y ago
  Cool, thank you for that tidbit of information!
• ColinHayhurst 4y ago
  OpenAI used Common Crawl for GPT-3. For GPT-4 they don't say anything.