4 ms·
> I am developing an evaluation suite so I can keep watching the progress in a systematic way.. Sounds like something that should be published on github
by taylodl 2y ago
> I am developing an evaluation suite so I can keep watching the progress in a systematic way..
Sounds like something that should be published on github
- pona-a 2y agoOpen benchmarks are vulnerable to saturation. I think benchmarks should have an embargo periodic, until which only 3% of the question-answer pairs is released, with an explicit warning not to use it 3 months after being released.