For training, yes, but these models are optimized for inference, since inference will be run many more times than training. The original Llama models were run w

For training, yes, but these models are optimized for inference, since inference will be run many more times than training. The original Llama models were run way past chinchilla-optimal amounts of data.