9 ms·
Correct
by tikkun 2y ago
Correct
- freediver 2y agoThat would make each API call cost at least $3 ($3 is price per million input tokens). And if you have a 10 message interaction you are looking at $30+ for the interaction. Is that what you would expect?
- rkwz 2y agoMaybe they're summarizing/processing the documents in a specific format instead of chatting? If they needed chat, might be easier to build using RAG?
- tr4656 2y agoThis might be when it's better to not use the API and just pay for the flat-rate subscription.
- coder543 2y agoGemini 1.5 Pro charges $0.35/million tokens up to the first million tokens or $0.70/million tokens for prompts longer than one million tokens, and it supports a multi-million token context window. Substantially cheaper than $3/million, but I guess Anthropic’s prices are higher.
- freediver 2y agoIt is also much worse.
- coder543 2y agoIs it, though? In my limited tests, Gemini 1.5 Pro (through the API) is very good at tasks involving long context comprehension. Google's user-facing implementations of Gemini are pretty consistently bad when I try them out, so I understand why people might have a bad impression about the underlying Gemini models.
- reitzensteinm 2y agoYou're looking at the pricing for Gemini 1.5 Flash. Pro is $3.50 for <128k tokens, else $7.
- coder543 2y agoAh... oops. For some reason, that page isn't rendering properly on my browser: https://imgur.com/a/XLFBPMI https://imgur.com/a/XLFBPMI When I glanced at the pricing earlier, I didn't notice there was a dropdown at all.
- impossiblefork 2y agoSo do it locally after predigesting the book, so that you have the entire KV-cache for it. Then load that KV-cache and add your prompt.