2 ms·

Has Meta disclosed how much parts of the dataset were repeated? I've only seen the "number of tokens trained" number.