I found this to be the key quote: > Since the 9B model is roughly 3 times smaller than the 27B model, at a fixed sampling compute budget we can sample 3× more

I found this to be the key quote: > Since the 9B model is roughly 3 times smaller than the 27B model, at a fixed sampling compute budget we can sample 3× more sample solutions per problem for Gemma2-9B. Essentially, training on samples from a weaker LLM is better than 1/3 the samples from a stronger LLM.