3 ms·
It looks like a pretty significant increase on SWE-Bench. Although that makes me wonder if there was some formatting or gotcha that was holding the results back
by ilaksh 2y ago
It looks like a pretty significant increase on SWE-Bench. Although that makes me wonder if there was some formatting or gotcha that was holding the results back before.
If this will work for your use case then it could be a huge discount versus o1. Worth trying again if o1-mini couldn't handle the task before. $4/million output tokens versus $60.
https://platform.openai.com/docs/pricing https://platform.openai.com/docs/pricing
I am Tier 5 but I don't believe I have access to it in the API (at least it's not on the limits page and I haven't received an email). It says "rolling out to select Tier 3-5 customers" which means I will have to wait around and just be lucky I guess.
- TechDebtDevin 2y agoGenuinely curious, What made you choose OpenAI as your preferred api provider? Its always been the least attractive to me.
- TeMPOraL 2y agoUntil recently they were the only game in town, so maybe they accrued significant spend back then?
- ilaksh 2y agoI have mainly been using Claude 3.5/3.6 Sonnet via API in the last several months (or since 3.5 Sonnet came out). However, I was using o1 for a challenging task at one point, but last I tested it had issues with some extra backslashes for that application. I also have tested with DeepSeek R1 and will test some more with that although in a way Claude 3.6 with CoT is pretty good. Last time I tried to test R1 their API was out.
- ipaddr 2y agoWho else might be a good choice? Deepseek is down. Who has the cheapest gpt3.5 level or above api
- TechDebtDevin 2y agoIve personaly been using Deepseek (which has been better than for 3.5 for a really long time), and Perplexity, which is nice for their built in search. Ive actually been using Deepseek since it was free. Its been generally good for me. Ive mostly chosen both because of pricing as I generally dont use APIs for extermely complex prompts.
- Aperocky 2y agoRun it locally, the distilled smaller ones aren't bad at all.
- eknkc 2y agoWe extensively used the batch APIs to decrease cost and handle large amount of data. I also need JSON responses for a lot of things and OpenAI seem to have the best json schema output option out there.
- yyhhsj0521 2y agoI use it because my company bought the enterprise plan and trained a bunch of "specialist GPTs" that were fed internal documents, code bases, Slack threads etc. I am not aware of other good LLM companies providing the same level of integration.
- TeMPOraL 2y agoTier 3 here and already see it on Limits page, so maybe the wait won't be long.
- ilaksh 2y agoYep, I got an email about o3-mini in the API an hour ago.
- TeMPOraL 2y agoI apparently got one at the same time too, but I missed it distracted by this HN thread :). Not only I got o3-mini (which I already noticed on the Limits page), but they also gave me access to o1 now! I'm Tier 3; until yesterday, o1 was still Tier 5 (IIRC). Thanks OpenAI! Nice gift and a neat distraction from DeepSeek-R1 - which I still can't use directly, because their API stopped working moments after I topped up my credits and generated an API key, and is still down for me... :/.
- sshh12 2y agoTier 5 and I got it almost instantly