Check out the graph on page 3 of this PDF: https://arxiv.org/abs/2302.13971 https://arxiv.org/abs/2302.13971 The 33B model started beating the 7B when it had be

Check out the graph on page 3 of this PDF: https://arxiv.org/abs/2302.13971 https://arxiv.org/abs/2302.13971 The 33B model started beating the 7B when it had been trained on only 1/3 as much data. And then they kept training it to 40% more than the total that 7B was trained on. It's better.