No. You need inference running in the user's session, sandboxed from inference running for some other user. Doubly so for RAG or agent control where the LLM is

No. You need inference running in the user's session, sandboxed from inference running for some other user. Doubly so for RAG or agent control where the LLM is expected to operate knobs and levers with the borrowed authority of the user. The LLM is just generating orders of magnitude more user state as a result of a prompt. The actions the LLM is permitted to take must still be gated on the authorizations allowed for that user. This means that data unavailable to the user must not be in the training set of the LLM acting as an interface layer. The "we have to learn new lessons" only applies when you lump all data together and hope the LLM doesn't spit up someone else's data from a probabilistic elbow jog. Hope is not a strategy.