2 ms·
Primary sources of AI training data
What are some of the primary sources of public data that are utilized in the training of large AI models?
- mindcrime 4y agoThe Facebook Llama paper[1] calls out a number of the ones they used. Briefly, they include: - CommonCrawl - Wikipedia - Github - StackExchange - arXiv and "Books" which appears to be made up from Project Gutenberg books among others. [1]: https://scontent-iad3-1.xx.fbcdn.net/v/t39.8562-6/333078981_693988129081760_4712707815225756708_n.pdf?_nc_cat=108&ccb=1-7&_nc_sid=ad8a9d&_nc_ohc=fskqjIsP1vwAX-_NNhB&_nc_ht=scontent-iad3-1.xx&oh=00_AfBUSnw_go9eRLVAfuVJ3BoItoxnsfO2qbFEC_FofFq1gA&oe=64216DE2 https://scontent-iad3-1.xx.fbcdn.net/v/t39.8562-6/333078981_...