The model trained 3 years ago was only trained on 300B tokens, heavily undertrained (in terms of the Chinchilla scale), that's why LLaMa models can easily beat

The model trained 3 years ago was only trained on 300B tokens, heavily undertrained (in terms of the Chinchilla scale), that's why LLaMa models can easily beat it on most benchmarks (they were trained on 1/1.4T tokens). About the current GPT-3.5 models, who knows, OpenAI is not very open about it.