2 ms·
How many stories/tokens did they actually train on? I can't find it in the paper.
by thomasahle 3y ago
How many stories/tokens did they actually train on? I can't find it in the paper.
- yyyk 3y agoMaybe this will help? https://huggingface.co/datasets/roneneldan/TinyStories https://huggingface.co/datasets/roneneldan/TinyStories Also note https://arxiv.org/abs/2309.05463 https://arxiv.org/abs/2309.05463 (which is larger - obviously size still does contribute to performance)