2 ms·
At least as far as what the GPT-3 papers claimed, all (or most?) of the data used for training would be freely available for other competitors/researchers to ac
by gorbypark 4y ago
At least as far as what the GPT-3 papers claimed, all (or most?) of the data used for training would be freely available for other competitors/researchers to acquire. Wikipedia, Common Crawl data, etc. I don’t believe OpenAI did their own crawling at all.
With OpenAI not being really open, it’s hard to say for sure what exactly ended up in the training materials, though. GPT-4 is even more of a black box to anyone outside of OpenAI with very little information released on how it was trained.