3 ms·
From your link: "We excluded reasoning models from our analysis of per-token prices. Reasoning models tend to generate a much larger number of tokens than othe
by MadxX79 7mo ago
From your link:
"We excluded reasoning models from our analysis of per-token prices. Reasoning models tend to generate a much larger number of tokens than other models, making these models cost more in total to evaluate on a benchmark. This makes it misleading to compare reasoning models to other models on price per token, at a given performance level."
It's just price per token. Token usage is exploding.
- simianwords 7mo agoToken usage is increasing but the link I shared is comparing price per token which is the reason they are excluding reasoning models. Reasoning models have also gotten cheaper. Generally cost to “achieve a certain task” using whatever model, even reasoning has drastically reduced. The best example is arc agi. https://arcprize.org/leaderboard https://arcprize.org/leaderboard This measures cost to achieve certain percentage of score. Fix a certain accuracy and see how much price reduces over time. It’s more than 100x.