I expect these responses to be just that, regurgitation. Case on point: I was playing using MRKL/ReACT like prompts with simple models like vanilla FLAN-T5 and

I expect these responses to be just that, regurgitation. Case on point: I was playing using MRKL/ReACT like prompts with simple models like vanilla FLAN-T5 and GPT-J. Clearly they didn't work as well as using GPT-3 (text-davinci-003), presumably due to being too small to present emergent abilities out-of-the-box and without finetuning. The funny thing is that when I prompted GPT-J with something like a ReACT prompt, it started responding with text that seemed to come straight out of a sci-fi novel about an AI gaining sentience. It is very amusing how "creative" these models "can be" sometimes.