My guess on "why optimization problems?": You can do self-play like AlphaZero against NP hard problems; easy to generate problems, easy to verify solutions, and

My guess on "why optimization problems?": You can do self-play like AlphaZero against NP hard problems; easy to generate problems, easy to verify solutions, and they will be probably be feeding the best meta-prompts and best solutions back into the fine-tuning once it's genericized enough to not be fine-tuning directly on the non-public benchmarks themselves. They also are trying to figure out what properties of prompts actually yield performance improvement/capabilities since that's still an open question. So far it's been highly subjective and qualitative and they've begun to systematize (via ablation) the effect of prompt structure. My guess is that the ultimate goal for this project is another meta-level or two that can lead directly to relatively stable goal-oriented self-improvement when the LLMs reach human-level ML engineering ability.