3 ms·
Asimov wrote a lot about what we could call alignment. I'd argue that our focus on benchmark based evaluations is akin to a misalignment with what we are actual
by godelski 3y ago
Asimov wrote a lot about what we could call alignment. I'd argue that our focus on benchmark based evaluations is akin to a misalignment with what we are actually trying to evaluate. While benchmarks are great tools and highly useful, the over-reliance of them is many a short story by Asimov.