3 ms·
They tested o1-preview but the results hidden away in the Appendix, probably because o1-preview's "drops" for 4 out of 5 of the new benchmarks are all within th
by famouswaffles 2y ago
They tested o1-preview but the results hidden away in the Appendix, probably because o1-preview's "drops" for 4 out of 5 of the new benchmarks are all within the margin of error.
i.e 94.9 on the full GSM8K and 93.6, 92.7, 95.4, 94 and 77.4 on the modified benchmarks.
The study proves nothing of the sort. Even the results of 4o are enough to give pause to this conclusion.