3 ms·
Can you please elaborate on how you prune?
by heliophobicdude 3y ago
Can you please elaborate on how you prune?
- vorticalbox 3y agoyou can just use the API, if you set completions to 0 it will return the token count. Then you can just remove the oldest message until it's under any number. I picked 3k to allow 1k for the reply
- akiselev 3y agoYou can also use the `tiktoken` python library: import tiktoken len(tiktoken.encoding_for_model("gpt-4").encode(contents))