4 ms·
For AI coding, o3-mini scored similarly to o1 at 10X less cost on the aider polyglot benchmark [0]. This comparison was with both models using high reasoning ef
by anotherpaulg 2y ago
For AI coding, o3-mini scored similarly to o1 at 10X less cost on the aider polyglot benchmark [0]. This comparison was with both models using high reasoning effort. o3-mini with medium effort scored in between R1 and Sonnet.
62% $186 o1 high
60% $18 o3-mini high
57% $5 DeepSeek R1
54% $9 o3-mini medium
52% $14 Sonnet
48% $0 DeepSeek V3
[0] https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/
- throw83288 2y agoWhat do you expect to come from full o3 in terms of automating software engineering?
- nextworddev 2y agoo3 (high) might score 80%+
- stavros 2y agoDo you have plans to try o3-mini-high as the architect and Sonnet as the model?
- seunosewa 2y agoYou're conflating the low price of the o3-mini medium effort model with the high performance of the o3-mini high effort model. OpenAI hasn't listed the price for the o3-mini high effort model separately on their pricing page.
- ZeroTalent 2y agoAlso Gemini API is free for coding.
- Alifatisk 2y agoI have yet to see a valid reason to use Gemini over any other alternative, the only exception being large contexts.
- dyauspitr 2y agoLarge context are a huge reason
- ZeroTalent 2y ago2 million tokens context windows separates it from all other LLMs.