4 ms·
It would be $1 for each individual API call, if you were continuing the conversation based on the same 100K input. ChatGPT is stateless.
by minimaxir 3y ago
It would be $1 for each individual API call, if you were continuing the conversation based on the same 100K input. ChatGPT is stateless.
- raylad 3y agoRight, so that adds up very fast.
- Der_Einzige 3y agoThis is a sad fact, and one which they should have implemented a fix for. We know medium term memory works. Sentence transformers and everyone playing with pooled embeddings knows what it is because they're using it. I should be able to map my previous history to a smaller number of tokens using embedding pooling to give a notion of a lossy "medium term" memory independent of RAG.