I don't think you necessarily need any validation signal to improve performance with synthetic data. Here's an example I posted further up: > Suppose you want

I don't think you necessarily need any validation signal to improve performance with synthetic data. Here's an example I posted further up: > Suppose you want to optimize the ability for an LLM to do arithmetic. You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results.