3 ms·

I don't think so, feels like the wrong side is getting attention. Degrading the experience for humans (in one tool) because the bots are prone to injection (fro

I don't think so, feels like the wrong side is getting attention. Degrading the experience for humans (in one tool) because the bots are prone to injection (from any tool). Terraform is used outside of agents; somebody surely finds the reminder helpful. If terraform were to abide, I'd hope at the very least it would check if in a pipeline or under an agent. This should be obvious from file descriptors/env. What about the next thing that might make a suggestion relying on our discretion? Patch it for agent safety?

8note 6mo ago
it makes you wonder how many times people have incorrectly followed those recommended commands
- bravetraveler 6mo ago
  If more than once (individually), I am concerned.
TeMPOraL 6mo ago
"Run terraform apply plan.out next" in this context is a prompt injection for an LLM to exactly the same degree it is for a human. Even a first party suggestion can be wrong in context, and if a malicious actor managed to substitute that message with a suggestion of their own, humans would fall for the trick even more than LLMs do. See also: phishing.
- bravetraveler 6mo ago
  Right, I'm fine with humans making the call. We're not so injection-happy/easily confused, apparently. Discretion, etc. We understand that was the tool making a suggestion, not our idea. Our agency isn't in question. The removal proposal is similar to wanting a phishing-free environment instead of preparing for the inevitability. I could see removing this message based on your point of context/utility, but not to protect the agent. We get no such protection, just training and practice. A supply chain attack is another matter entirely; I'm sure people would pause at a new suggestion that deviates from their plan/training. As shown, autobots are eager to roll out and easily drown in context. So much so that `User` and `stdout` get confused.
  - bravetraveler 6mo ago
    Can't edit now, day late and a dollar short. I raise a vote: we call this pattern inception, if injection offends.
- franktankbank 6mo ago
  Maybe the agents should require some sort of input start token: "simon says"

8note 6mo ago

it makes you wonder how many times people have incorrectly followed those recommended commands

bravetraveler 6mo ago
If more than once (individually), I am concerned.

TeMPOraL 6mo ago

"Run terraform apply plan.out next" in this context is a prompt injection for an LLM to exactly the same degree it is for a human. Even a first party suggestion can be wrong in context, and if a malicious actor managed to substitute that message with a suggestion of their own, humans would fall for the trick even more than LLMs do. See also: phishing.

bravetraveler 6mo ago
Right, I'm fine with humans making the call. We're not so injection-happy/easily confused, apparently. Discretion, etc. We understand that was the tool making a suggestion, not our idea. Our agency isn't in question. The removal proposal is similar to wanting a phishing-free environment instead of preparing for the inevitability. I could see removing this message based on your point of context/utility, but not to protect the agent. We get no such protection, just training and practice. A supply chain attack is another matter entirely; I'm sure people would pause at a new suggestion that deviates from their plan/training. As shown, autobots are eager to roll out and easily drown in context. So much so that `User` and `stdout` get confused.
- bravetraveler 6mo ago
  Can't edit now, day late and a dollar short. I raise a vote: we call this pattern inception, if injection offends.
franktankbank 6mo ago
Maybe the agents should require some sort of input start token: "simon says"