2 ms·
I've not tested its limits yet. The thing to consider is prompt length - depending on the model you get around 4,000 tokens, and the prompt itself is already 26
by simonw 4y ago
I've not tested its limits yet. The thing to consider is prompt length - depending on the model you get around 4,000 tokens, and the prompt itself is already 264 according to https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer - you need a bunch of space left for providing the output of your various actions, so tokens get used up pretty quickly.
The ReAct paper talks about fine-tuning to teach a model actions. I'd be interested to see an experiment that fine-tunes the LLaMA model to teach it actions - I have a hunch that might work really well, and save a bunch of token space in the actual execution phase.