3 ms·
LiveBench (which I like because it tries very hard to avoid contamination) ranks Sonnet 3.5 second only to o1 (which is totally expected).
by thegeomaster 2y ago
LiveBench (which I like because it tries very hard to avoid contamination) ranks Sonnet 3.5 second only to o1 (which is totally expected).
- behnamoh 2y agono, sonnet 3.5 is #7 on LiveBench, even below DeepSeek V3.
- thegeomaster 2y agoThe parent comment was talking about coding specifically, not the average score. I see o1 at 69.69, and Claude 3.5 Sonnet at 67.13.
- sebastiennight 2y agoo1's score looks like exactly what I would expect Elon Musk to aim for with Grok's benchmarks
- parav 2y agoLiveCodingBench has DeepSeekR1 at #3 after O1-high and O1-medium https://livecodebench.github.io/leaderboard.html https://livecodebench.github.io/leaderboard.html