4 ms·
The lack of distinction is confusing but Apple have released both the training source and the inference source + models/weights at the same time — so at least b
by gilgoomesh 2y ago
The lack of distinction is confusing but Apple have released both the training source and the inference source + models/weights at the same time — so at least both are true.
- nicce 2y agoIs the training data public as well?
- raxxorraxor 2y agoThis time Apple could say that they didn't release it for privacy reasons and this time it would be correct for once.
- GeekyBear 2y agoYes > RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text documents coming from 84 CommonCrawl snapshots and processed using the CCNet pipeline. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data > Dolma Dataset: an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. https://allenai.github.io/dolma/ https://allenai.github.io/dolma/