3 ms·
>However, they did not publish any details on their training data. Yes they did. It's in the README.
by charcircuit 4y ago
>However, they did not publish any details on their training data.
Yes they did. It's in the README.
- option 4y agoAll I can see is “ 1.7 TB of online texts, books, and countless other sources in both English and Russian.” if there are more details, can you please share a link? I am worried that “other sources” may contain Yandex.news which is a cesspool of anti-West and anti-Ukraine propaganda
- 533474 4y agoThe pile dataset is used for the English language
- remram 4y agoDirect link: https://github.com/yandex/YaLM-100B/blob/main/README.md#training-details https://github.com/yandex/YaLM-100B/blob/main/README.md#trai...