4 ms·
which is a good hint that that benchmark sucks. No way 4o beats sonnet 3.5
by nunodonato 2y ago
which is a good hint that that benchmark sucks. No way 4o beats sonnet 3.5
- lauralex 2y agoSonnet 3.5 has a lot of alignment issues. It many times refused to answer simple coding questions I asked, just because it considered them "unsafe". 4o is much more relaxed. Regarding math, sonnet is a bit better than 4o though.
- ashu1461 2y agoI think they have secretly released something which is better than 4. In our internal benchmarks also the 4 o mini is performing better than 4 o
- rvnx 2y agoThe weirdest is that ultimately the best model is supposed to be Gemini Pro according to these benchmarks