4 ms·
Sure, running an LLM is cheaper, but the way we use LLMs now requires way more tokens than last year.
by InsideOutSanta 9mo ago
Sure, running an LLM is cheaper, but the way we use LLMs now requires way more tokens than last year.
- simianwords 9mo agook but the capabilities are also rising. what point are you trying to make?
- oytis 9mo agoThat it's not getting cheaper?
- jstummbillig 9mo agoBut it is, capability adjusted, which is the only way it makes sense. You can definitely produce last years capability at a huge discount.
- simianwords 9mo agoyou are wrong. https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends this is accounting for the fact that more tokens are used.
- techpression 9mo agoThe chart shows that they’re right though. Newer models cost more than older models. Sure they’re better but that’s moot if older models are not available or can’t solve the problem they’re tasked with.
- simianwords 9mo agothis is incorrect. the cost to achieve the same task by old models is way higher than by new models. > Newer models cost more than older models where did you see this?
- techpression 9mo agoOn the link you shared, 4o vs 3.5 turbo price per 1m tokens. There’s no such thing as ”same task by old model”, you might get comparable results or you might not (and this is why the comparison fail, it’s not a comparison), the reason you pick the newer models is to increase chances of getting a good result.
- simianwords 9mo ago> The dataset for this insight combines data on large language model (LLM) API prices and benchmark scores from Artificial Analysis and Epoch AI. We used this dataset to identify the lowest-priced LLMs that match or exceed a given score on a benchmark. We then fit a log-linear regression model to the prices of these LLMs over time, to measure the rate of decrease in price. We applied the same method to several benchmarks (e.g. MMLU, HumanEval) and performance thresholds (e.g. GPT-3.5 level, GPT-4o level) to determine the variation across performance metrics This should answer. In your case, GPT-3.5 definitely is cheaper per token than 4o but much much less capable. So they used a model that is cheaper than GPT-3.5 that achieved better performance for the analysis.
- fooker 9mo agoOpenAI has always priced newer models lower than older ones.
- techpression 9mo agohttps://platform.openai.com/docs/pricing https://platform.openai.com/docs/pricing Not according to their pricing table. Then again I’m not sure what OpenAI model versions even mean anymore, but I would assume 5.2 is in the same family as 5 and 5.2-pro as 5-pro
- fooker 9mo agoCheck GPT 5.2 vs it's predecessor the 'o' series of reasoning models.
- simianwords 9mo agonot true! 4o was costlier than 3.5 turbo
- fooker 9mo ago10x more tokens today cost less than than half of X tokens from ~mid 2024.