2 ms·
Wouldn't work. A dynamic but verifiable problem. Is just a perfect target for a RL environment. If you don't have the verifiable part the benchmark is useless,
by gunalx 23d ago
Wouldn't work. A dynamic but verifiable problem. Is just a perfect target for a RL environment. If you don't have the verifiable part the benchmark is useless, or really expensive with human review. (Or just open ended)
- dpweb 23d agoThere doesn't need to be a single correct response. Generate "novel" problems with a set of acceptable solutions/outcomes, then verify that the response satisfies the criteria.