3 ms·
I am interested to use the assistant api for my commercial project but it is not clear from the article what the token count looks like? - is it counted for a
by jerrygoyal 3y ago
I am interested to use the assistant api for my commercial project but it is not clear from the article what the token count looks like?
- is it counted for a single user message or the sum of all previous messages?
- if there's a file, will it be counted every time a user interacts or only the first time?
- visarga 3y agoI think - it is correlated to the sum, every new interaction adds the whole history again - yes, but you probably pay for the retrieved fragments, not the whole file
- brandall10 3y agoOn the second point, there was an issue on launch where it would not find a relevant fragment and appear to load the whole file into the context. Unsure if this has changed but it freaked quite a few folks out OpenAI discussion forums w/ escalating costs.