depending on the max tokens, I think you can pretty easily fine-tune a model to return answers with actions required then wrap your prompt app to react to those

depending on the max tokens, I think you can pretty easily fine-tune a model to return answers with actions required then wrap your prompt app to react to those, paste answers and "reask/reprompt" the same question... similar stuff is being research under "langchains" term