3 ms·
> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy" I don't believe Cerebras has a cached inp
by peri-cl 1mo ago
> "Also maybe my setup (OMP) doesn't do the cache correctly but that's a huge cost driver... so atm it's quite pricy"
I don't believe Cerebras has a cached input pricing? They don't list one on the model page:
https://inference-docs.cerebras.ai/models/qwen-3.8-27b https://inference-docs.cerebras.ai/models/qwen-3.8-27b
edit: See the sibling discussion,
https://news.ycombinator.com/item?id=49554520#49555094 https://news.ycombinator.com/item?id=49554520#49555094 ("Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate")
- olivermuty 1mo agoThey have cache, but it costs the same indeed, no idea what the point of the cache is
- lostmsu 1mo agoThey don't have cache (e.g. KV cache). But they write down what you sent earlier to say they cached it! To still bill the same as uncached later (because they didn't actually cache it)!
- orbifold 1mo agoMore precisely they can't cache it.
- lostmsu 1mo agoYes. Their architecture recomputes every time so at 150k context every request will have to spend 1.5 min waiting for the model to reread the context. Say avg model response length is 1024 tok. At 50 tok/s normal providers do your turn will only take 20s (vs Cerebras 101s) and will cost 20x less. That time and cost is per single tool call.
- hexa00 1mo agolol yeah just saw that, yeah that makes it unusable I think at least for me. I wonder if they will do that with sol ultrafast!