5 ms·
> ~=$3400 per single task report says it is $17 per task, and $6k for whole dataset of 400 tasks.
by riku_iki 2y ago
> ~=$3400 per single task
report says it is $17 per task, and $6k for whole dataset of 400 tasks.
- bluecoconut 2y agoThat's the low-compute mode. In the plot at the top where they score 88%, O3 High (tuned) is ~3.4k
- ionwake 2y agosorry to be a noob, but can someone tell me doe sths mena o3 will be unaffordable for a typical user? Will only companies with thousands to spend per query be able to use this? Sorry for being thick Im just confused how they can turn this into an addordable service?
- JohnnyMarcone 2y agoThere are likely many efficiency gains that will be made before it's released, and after. Also they showed o3 mini to be better than o1 for less cost in multiple benchmarks, so there're already improvements there at a lower cost than what available.
- ionwake 2y agoGreat thank you
- HDThoreaun 2y agoThe low compute one did as well as the average person though
- jhrmnn 2y agoThat’s for the low-compute configuration that doesn’t reach human-level performance (not far though)
- riku_iki 2y agoI referred on high compute mode. They have table with breakdown here: https://arcprize.org/blog/oai-o3-pub-breakthrough https://arcprize.org/blog/oai-o3-pub-breakthrough
- EVa5I7bHFq9mnYK 2y agoThat's high EFFICIENCY. High efficiency = low compute.
- gbnwl 2y agoThat's "efficiency" high, which actually means less compute. The 87.5% score using low efficiency (more compute) doesn't have cost listed.
- bluecoconut 2y agothey use some poor language. "High Efficiency" is O3 Low "Low Efficiency" is O3 High They left the "Low efficiency" (O3 High) values as `-` but you can infer them from the plot at the top. Note the $20 and $17 per task aligns with the X-axis of the O3-low
- junipertea 2y agoThe table row with 6k figure refers to high efficiency, not high compute mode. From the blog post: Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration.
- deleted 2y ago[deleted]
- binarymax 2y ago"Note: OpenAI has requested that we not publish the high-compute costs. The amount of compute was roughly 172x the low-compute configuration." The low compute was $17 per task. Speculate 172*$17 for the high compute is $2,924 per task, so I am also confused on the $3400 number.
- bluecoconut 2y ago3400 came from counting pixels on the plot. Also its $20 on for the o3-low via the table for the semi-private, which x172 is 3440, also coming in close to the 3400 number
- xrendan 2y agoYou're misreading it, there's two different runs, a low and a high compute run. The number for the high-compute one is ~172x the first one according to the article so ~=$2900
- Thorrez 2y agoWhat's extra confusing is that in the graph the runs are called low compute and high compute. In the table they're called high efficient and low efficiency. So the high and low got swapped.