3 ms·
But the prompt didn't give the models the option to say "I don't know", so it wasn't a measure of their confidence.
by simonw 5mo ago
But the prompt didn't give the models the option to say "I don't know", so it wasn't a measure of their confidence.
- zug_zug 5mo agoI mean that's true but I don't think that's realistically what's going on when one model gives an unqualified "Yes" and the other gives an unqualified "no." You can argue the study isn't as case-closed-decisive as we'd ideally like, but it's certainly evidence. It's probably hard to design a better study.