TLDW: Prompt injection. It's not a bug, it's a fundamental feature, you can mitigate but not "fix" it. The LLM is just a Make Document Bigger algorithm being

TLDW: Prompt injection. It's not a bug, it's a fundamental feature, you can mitigate but not "fix" it. The LLM is just a Make Document Bigger algorithm being run on a single document. All one stream of words/tokens. The algorithm does not firmly track which parts of the document are trusted, or recognize entities, or recognize local propositions from a particular entity, or know whether the proposition is being asserted or just quoted, etc.