If I give someone a piece of code, they can prove things about the code, even if it is at a higher level of abstraction. A prompt isn't an abstraction. Say you

If I give someone a piece of code, they can prove things about the code, even if it is at a higher level of abstraction. A prompt isn't an abstraction. Say you evaluate an llm completely deterministically. Fixed seed for the sampling, or maybe zero temperature. You give the prompt to an engineer and ask them what it does. He can't really say for sure. 'Try "compiling it". It doesn't work? Well try adjusting your prompt a bit. How? I have no idea; try renaming some variable to shirley or something see what pops out.'