3 ms·
Cheers, Simon - been seeing your comments around and enjoying your blog and coverage of this stuff. Is that prompt in your TIL really all it takes to inform it
by nighthawk454 4y ago
Cheers, Simon - been seeing your comments around and enjoying your blog and coverage of this stuff.
Is that prompt in your TIL really all it takes to inform it of these 3 actions? That's pretty impressive. I wonder how many actions it can scale to? I kind of expected some kind of classifier layer to predict if an action was necessary!
- simonw 4y agoI've not tested its limits yet. The thing to consider is prompt length - depending on the model you get around 4,000 tokens, and the prompt itself is already 264 according to https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer - you need a bunch of space left for providing the output of your various actions, so tokens get used up pretty quickly. The ReAct paper talks about fine-tuning to teach a model actions. I'd be interested to see an experiment that fine-tunes the LLaMA model to teach it actions - I have a hunch that might work really well, and save a bunch of token space in the actual execution phase.