3 ms·
No, the purpose was to create a (automated) test set in the first place. The author builds an LLM judge which can score the LLMs participating during test-time.
by isomorphic_duck 3mo ago
No, the purpose was to create a (automated) test set in the first place. The author builds an LLM judge which can score the LLMs participating during test-time. That would be why the author used the strongest model (Opus 4,7 at the time) as the judge.
- brigandish 3mo agoYes, by giving hints it verifies the test because if it can't find it with hints then it shouldn't find it without hints, else there is some other mechanism at work (which would also be useful to know, like access to other tools).