4 ms·
>so you will need to truncate the beginning of the conversation at some point I wrote a little python script to keep track of a running conversation when I sta
by knome 4y ago
>so you will need to truncate the beginning of the conversation at some point
I wrote a little python script to keep track of a running conversation when I started playing with openai's completions API. I keep track of how many tokens the prompt is taking up, and when it gets too close to some configurable threshold, I then have a different prompt to tell the AI to summarize the conversation and any previous summary, with various demands to keep track of certain specific bits of state ( like names and things ).
Works pretty well for marching conversations past the token limit.
- davedx 4y agoFascinating, and somewhat comparable to how the human brain "compresses" visual scenes when encoding them into long term memory. This is such an interesting space to follow :)
- pixl97 4y agoHas anyone played around with GPT itself to see if you can get it to track the number of tokens it's tracking (best if implemented with the least number of tokens possible)
- lasermatts 4y agoI haven't played too much with doing this programmatically, but OpenAI's Tokenizer tool is a fun way to get an intuition around tokens and text length https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
- singularity2001 4y agosimple but brilliant, should be the default behavior for all token limited agents!
- davedx 4y agoI honestly keep thinking about this approach and talking about it to other people I know. If you don't mind sharing the code I'd be super interested to see how it works