You can take a look at chinchilla or Llama scaling laws. It depends on what you want to optimize. If you're interested in inference optimal, go with llama, if

You can take a look at chinchilla or Llama scaling laws. It depends on what you want to optimize. If you're interested in inference optimal, go with llama, if it's compute optimal, go with chinchilla. GPT-4 technical paper also has good scaling laws for how benchmarks will behave given extrapolating from a small training set.