If I understand correctly what you are imagining is some sort of local retrieval enhancement to generate a specific part of the overall response, which later is

If I understand correctly what you are imagining is some sort of local retrieval enhancement to generate a specific part of the overall response, which later is removed, once e.g. the generated piece of code (e.g. a function call) has completed generation. Indeed, this is a form of LLM prompting that we are also exploring in our preview release channel with something called in-context functions. See https://next.lmql.ai https://next.lmql.ai and choose the "In-Context Functions" example in the New Features showcase screen. With in-context functions, you can provide additional instructions/data to the LLM, that will apply locally only. Once such an in-context function returns, additional instructions/retrieved info is removed and only the LLM-generated end-result remains.