3 ms·
Ed's main thesis is that cost is unsustainable for AI companies but this is clearly wrong. The unit cost is going down and has gone down by more than 20-30x ov
by simianwords 8mo ago
Ed's main thesis is that cost is unsustainable for AI companies but this is clearly wrong.
The unit cost is going down and has gone down by more than 20-30x over the years. Sure, the fixed cost of training is going up but that's because of the implied returns. Once the returns to training don't happen, it would simply reduce modulo cutoff date updates. The companies have a choice to just stop training and focus on inference cost reduction.
What am I missing here? Unless the consumers decide that they are no longer willing to pay the same amount as before and their expectations are rising with prices falling, what else?
- MadxX79 8mo agoIs that the cost per token or the actual cost of the user having a conversation, reasoning and all?
- simianwords 8mo agoCost per defined capability. Meaning you fix the task and then find how much it cost to achieve it including reasoning, tokens etc.
- MadxX79 8mo agosource?
- simianwords 8mo agohttps://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends
- MadxX79 8mo agoFrom your link: "We excluded reasoning models from our analysis of per-token prices. Reasoning models tend to generate a much larger number of tokens than other models, making these models cost more in total to evaluate on a benchmark. This makes it misleading to compare reasoning models to other models on price per token, at a given performance level." It's just price per token. Token usage is exploding.
- simianwords 8mo agoToken usage is increasing but the link I shared is comparing price per token which is the reason they are excluding reasoning models. Reasoning models have also gotten cheaper. Generally cost to “achieve a certain task” using whatever model, even reasoning has drastically reduced. The best example is arc agi. https://arcprize.org/leaderboard https://arcprize.org/leaderboard This measures cost to achieve certain percentage of score. Fix a certain accuracy and see how much price reduces over time. It’s more than 100x.