4 ms·
Yesterday I did some testing on the cost to solve the same simple problem on openrouter with different models using cline. Simple problem but it had a few nuanc
by elcritch 5mo ago
Yesterday I did some testing on the cost to solve the same simple problem on openrouter with different models using cline. Simple problem but it had a few nuances to solve it properly and so required reasoning.
After reading comments like this I was expecting (hoping?) that DeepSeek or similar would be cheaper.
However I was surprised that DeepSeek v4 cost about 5.5x GPT-5.4 to solve the problem.
- Deepseek-v4-pro-medium cost $2.47
- GPT-5.4-medium cost $0.45
- GPT-5.5-low was $0.86
- VulgarExigency 5mo agoThat doesn't sound right. Were you using the actual Deepseek provider? The one time I spent 3 dollars on Deepseek in a day, I had 615k output tokens, 96M cache hit input tokens, and 5M cache miss output tokens.
- HDBaseT 5mo agoYeah, I struggle to use more than a few dollars a day using Deepseek V4 Pro (max reasoning). * Some people suggest not using max reasoning due to overthinking and looping issues, this may consume more tokens than needed.
- nl 5mo agoIt's not unheard of for "more expensive" models (on a per-token basis) to end up cheaper than weaker models (on a per-task basis). Kimi K2.5 is roughly double the price (per token) of DeepSeek v4 Pro, but cost $0.05 vs $0.16 (for the same score) on my own benchmark. https://sql-benchmark.nicklothian.com/?highlight=moonshotai_kimi-k2.5 https://sql-benchmark.nicklothian.com/?highlight=moonshotai_... https://sql-benchmark.nicklothian.com/?highlight=deepseek_deepseek-v4-pro https://sql-benchmark.nicklothian.com/?highlight=deepseek_de...