3 ms·
It really shows how good Deepseek R1 is (even though it was evaluated only on text-only questions). The results are shown here: https://lastexam.ai/ https://la
by GaggiX 2y ago
It really shows how good Deepseek R1 is (even though it was evaluated only on text-only questions).
The results are shown here: https://lastexam.ai/ https://lastexam.ai/
EDIT: the text-only evaluation of the models shown in the paper gives o1 an accuracy of 8.9%, so Deepseek R1 is even better than I thought.
- myrmidon 2y agoIs there a text-only evaluation of the non-Deepseek models? Because being evaluated on text-only might have helped the other models immensely as well from what I can tell?
- GaggiX 2y ago>Is there a text-only evaluation of the non-Deepseek models? Not that I can see but it would be cool to have, maybe the paper will a more complete evaluation.
- famouswaffles 2y agoSection C.2 of the paper (pg 24) has text only evaluations of other models.
- GaggiX 2y agoOh I see, the paper is out, I read "(arXiv coming soon)" and though it wasn't released yet.