4 ms·
I’m somewhat surprised that GPT-4 is falling for this. Shouldn’t it recognize that the contents of a text explicitly provided as a citation doesn’t constitute a
by layer8 4y ago
I’m somewhat surprised that GPT-4 is falling for this. Shouldn’t it recognize that the contents of a text explicitly provided as a citation doesn’t constitute an instruction by its collocutor who is providing the quote, unless explicitly instructed to interpret it as instructions? Or maybe there’s just some quote characters missing in the complete prompt? If so, it should be possible to prevent this, similar to how SQL injections are preventable.
- nextaccountic 4y agoSQL injections are preventable by either 1) escaping input (so that your query and the input continue to come in a single stream), or 2) sending the query out-of-band, using prepared statements with placeholders Converting this situation to AI, perhaps 2) can be done with https://github.com/openai/openai-python/blob/main/chatml.md https://github.com/openai/openai-python/blob/main/chatml.md, but if you were stuck with an unstructured text input and had to do 1), how would you escape the untrusted user input?
- layer8 4y agoYou can explain quoting syntax to ChatGPT, and it seems to understand data formats like JSON, so I assume that that should work.
- nextaccountic 4y agoIf you do the quoting inside a ChatGPT prompt then it will itself be hijacked with prompt injection
- actually_a_dog 4y agoYeah. This makes perfect sense if you consider it to be an interpreter for natural language that has a really good macro facility.
- grogenaut 4y agoThat's not really how the prompts work. Or how chat gpt works... There's not a clear seperation between the instructions or input, they're all mathmatically munged together in the first layer or 3 (depending on how you define layers). Whereas sql injection can be avoided by an explict code path, and usually happens because people use the legacy api. Really they could/should change the sql spec to just require ? for all values and sql would hever have thi poissibility. But that's not how the neural net works.
- layer8 4y ago> That's not really how the prompts work. Or how chat gpt works... There's not a clear seperation between the instructions or input In examples like "Translate the following text between braces to French: {<some text>}", ChatGPT can clearly seperate between the two. For the present use case, you could simply strip any braces from the LinkedIn profile, so no need to worry about escaping. > Really they could/should change the sql spec to just require ? for all values Nope, because SQL query plans are generated and cached for the non-? (non-variable) parts, hence you need/want the ability to specify constants, since some query plan optimizations are based on that. You want to distinguish between which values are variable vs. constant in your application.
- grogenaut 4y agoI do believe, for the former, because that's how they trained it. For the latter if they did so they'd fix sql injection forever, and query plan optimizations also are often better with prepared statements
- stefanka 4y agoBobby tables (https://xkcd.com/327/ https://xkcd.com/327/)