5 ms·
One huge problem with these "cheap" models is that they happen to be more expensive in the typical agent workflow if the provider does not support caching. Inp
by grim_io 1y ago
One huge problem with these "cheap" models is that they happen to be more expensive in the typical agent workflow if the provider does not support caching.
Input and output costs are peanuts compared to the order of magnitude(or more) amount of tokens that hit the cache.
At that point you might as well use GPT-5. It will be the same price or cheaper, and more capable.
- NotMichaelBay 1y agoI was under the impression that this model does support caching. The pricing page says the cost of input tokens (cache hit) is $0.028.
- segmondy 1y agoyou declared a huge problem and followed up with an IF. deepseek API supports caching, stop manufacturing problems where there is none. https://api-docs.deepseek.com/guides/kv_cache https://api-docs.deepseek.com/guides/kv_cache
- grim_io 1y agoSure. But there is no way I'm going to use the deepseek endpoint. Openrouter says they might use your data for training.
- cheema33 1y agoFirst you complained about lack of caching. When you were informed that the model supports caching, instead of admitting your error you switched to an unrelated complaint. I hope that you you do not use similar strategies for discussion in your personal and work life.
- grim_io 1y agoYour broad attack on me as a person is unnecessary. If you read my post carefully, you will realize that I did not make any contradictory statements.
- segmondy 1y agocaching is not a function of the model but the provider, all models can be cached. the provider serving the model decides if they are going to cache it. openrouter is not a provider but a middleman between providers, so some of their providers for deepseek might provide caching and some might not. if you just use any then you might run into the issue. some of their provider might use your data for training, some might not. you have to look at the list and you can cherry pick ones that won't train on your data and that also provide caching.
- JimDabell 1y ago> One huge problem with these "cheap" models is that they happen to be more expensive in the typical agent workflow if the provider does not support caching. DeepSeek supports caching and cache hits are a tenth of the cost. $0.028/M for cache hit $0.28/M for cache miss $0.42/M for output — https://api-docs.deepseek.com/news/news250929 https://api-docs.deepseek.com/news/news250929
- grim_io 1y agoI auto disqualify the chinese first party endpoints. If they are okay for you, then sure go ahead. Enjoy the caching. What other provider is going to support it?
- JimDabell 1y ago> I auto disqualify the chinese first party endpoints. Why?
- curseofcasandra 1y agoI’m guessing it’s something along the lines of this: https://youtu.be/kYiUY07TzS4 https://youtu.be/kYiUY07TzS4
- guluarte 1y agoby your logic then you have to disqualify openai and anthropic first party endpoints for testing gpt and claude...
- grim_io 1y agoThere is no bug in my logic. Anthropic and OpenAI are not chinese first party providers.