Think of the LLM as an interpreter, and the initial prompt as a script. When you reply, it appends the script and reruns the program. Because it’s basically a

Think of the LLM as an interpreter, and the initial prompt as a script. When you reply, it appends the script and reruns the program. Because it’s basically a giant word frequency/relationship chart, you can overwhelm the initial prompt by either logic or by word frequency. If the initial prompt is 100 characters and you have 500 characters of different instruction, you basically overflow the original script with yours. On the formal logic side you can find different ways to retroactively comment out the first part of the script. It’s a hard problem to fix because of how these things work. Black boxes fed text files to parse.