23 ms·
OpenAI/evals > Building an eval: https://github.com/openai/evals/blob/main/docs/build-eval.md https://github.com/openai/evals/blob/main/docs/build-eval.md "Rob
by westurner 3y ago
OpenAI/evals > Building an eval: https://github.com/openai/evals/blob/main/docs/build-eval.md https://github.com/openai/evals/blob/main/docs/build-eval.md
"Robustness of Model-Graded Evaluations and Automated Interpretability" (2023)
https://www.lesswrong.com/posts/ZbjyCuqpwCMMND4fv/robustness-of-model-graded-evaluations-and-automated https://www.lesswrong.com/posts/ZbjyCuqpwCMMND4fv/robustness... :
> The results inspire future work and should caution against unqualified trust in evaluations and automated interpretability.
From https://news.ycombinator.com/item?id=37451534 https://news.ycombinator.com/item?id=37451534 : add'l benchmarks: TheoremQA, Legalbench