LLaMa 3.1 has been pre-trained on 15 trillion tokens, plus some more millions for the fine-tuning. About 60 terabytes. https://github.com/meta-llama/llama-mode

LLaMa 3.1 has been pre-trained on 15 trillion tokens, plus some more millions for the fine-tuning. About 60 terabytes. https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md https://github.com/meta-llama/llama-models/blob/main/models/... The heaviest quantised LLaMa 3.1 8B is about 3.4GB. So 0.005% compression rate, if you don't mind the intelligence of a heavily quantised 8B model.