A lot of the interest in LLMs is as a component in a larger software system that actually does something. So for example, your product might just present a tex

A lot of the interest in LLMs is as a component in a larger software system that actually does something. So for example, your product might just present a text box where you type in something in your preferred language, and then the LLM translates it and sends it back as a JSON object that can be sent to your phone and simultaneously change the display language of the app to the language you're speaking. The developer sets up a "prompt" for this (the first command), and then the user enters the actual data that the LLM operates on. The problem the article points out is that there's no separate sandbox for prompts vs. data. To the LLM, it's all just a conversation. So a sufficiently savvy user can alter the context that the LLM frames all subsequent responses with - in this case, to talk like a pirate. And if that output is being fed into downstream systems, they can use the LLM to trick the downstream systems into doing things they shouldn't be able to do.