Even taking a simple system that is supposed to summarize long texts that might exceed the context size: the simple approach is to cut the document into segment

Even taking a simple system that is supposed to summarize long texts that might exceed the context size: the simple approach is to cut the document into segments, have the LLM summarize each segment separately, then generate a summary of those summaries. Now you have to defend against attacks not just from the original text, but also from the intermediate summaries (which are indirectly under attacker control). Which is only going to get worse as we add more opportunities for internal thought to our models, which also has to be protected. It's like defending against SQL injection before parameterized statements were invented. Forget calling real_escape_string(input) once in your entire codebase, and the attacker owns your system.