4 ms·
An interesting parameter that I don't read about a lot is vocab size. A larger vocab means you will need to generate less tokens for the same word on average, a
by sakex 2y ago
An interesting parameter that I don't read about a lot is vocab size. A larger vocab means you will need to generate less tokens for the same word on average, also the context window will be larger. This means that a model with a large vocab might be more expensive on a per token basis, but would generate less tokens for the same sentence, making it cheaper overall. This should be taken into consideration when comparing API prices.
- weird-eye-issue 2y agoYeah... it obviously uses the appropriate tokenizer
- spencerchubb 2y agoSome companies like Anthropic haven't publicly released the tokenizer used in their API, thereby making it impossible for this library to use the appropriate tokenizer in all cases. Be careful about how you use the word 'obviously'
- deleted 2y ago[deleted]
- neverokay 2y agoGemini also only charges for output tokens, not sure if that’s considered. All in all this is something I was looking for or was roughly going to do to compare costs. Cool stuff.