I think an important baseline question is how much you care when the user of the LLM forces your model to divulge sensitive details or large chunks of that data

I think an important baseline question is how much you care when the user of the LLM forces your model to divulge sensitive details or large chunks of that data. Imagine that the LLM portion is running as client-side browser JavaScript application: Nothing in the training data or prompt is reliably-secret, and a determined user can get it to emit almost anything they like to whatever is downstream.