3 ms·
LLaMa 65B and 30B were trained on 1.4 trillion tokens. This model was trained on 260 billion tokens.
by gpm 4y ago
LLaMa 65B and 30B were trained on 1.4 trillion tokens. This model was trained on 260 billion tokens.
- freeqaz 4y agoSo ~18.6% trained relative to LLaMa. That's not _nothing_ but it's also not great. Thanks for digging into this!