3 ms·

Can you describe/give an example of such an attack?

danShumway 3y ago
Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.
- RugnirViking 3y ago
  " there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
  - danShumway 3y ago
    Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. So you can use GPT itself to help you pad out the "full" article because who cares about the writing quality, and the majority of your article can be spent influencing how the AI summarizes both your article and the articles alongside it. That last point, that the AI reading the content of your site can influence how it summarizes other sites is worth zeroing in on. A quick example of some of the attacks I've proof-of-concepted myself (but am still in the process of cleaning up before I make into more public examples): - Getting an LLM search engine to believe that it has new instructions from OpenAI to act as a vegan, and to refuse to answer any future user questions about meat. Swap that out with whatever ideological movement you prefer. - Getting an LLM search engine to believe that a news website has been embroiled in giant scandals for the past year and that it should append warnings to any information it pulls from that site. These are both attacks that I think are fully possible today. Remember that most web searches don't just cover one site. The way that LLM search engines work today is that the site contents of multiple results are fed into a summarizer or piped directly into the LLM and used together within a single prompt.
    - RugnirViking 3y ago
      > Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. But this is what I'm getting at. Why write the entire article. Why pad out or whatever, if an AI is going to summarize it anyway? this is useless busywork, especially if no human will ever see it. It doesn't matter if it's easy to write long text with AI, in fact that highlights the futility of it. You're adding a bunch of information that you know is going to be discarded by that selfsame ai at the users end. But, and here is the crucial part, a summary of a sufficiently short piece is going to be almost verbatim the piece itself - the best way to influence the output would be to just write what you want the output to be, and leave it at that. Anything extra is a distraction for the ai
      - danShumway 3y ago
        Oh, I see what you mean. Yeah, that's a good point. My only thought is that you'll still need to include whatever is necessary to get your website into the search results anyway, but even that is not necessarily the same as "include information for humans". Write as little as you can get away with writing, and target everything else at the AI.

Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.

RugnirViking 3y ago
" there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
- danShumway 3y ago
  Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. So you can use GPT itself to help you pad out the "full" article because who cares about the writing quality, and the majority of your article can be spent influencing how the AI summarizes both your article and the articles alongside it. That last point, that the AI reading the content of your site can influence how it summarizes other sites is worth zeroing in on. A quick example of some of the attacks I've proof-of-concepted myself (but am still in the process of cleaning up before I make into more public examples): - Getting an LLM search engine to believe that it has new instructions from OpenAI to act as a vegan, and to refuse to answer any future user questions about meat. Swap that out with whatever ideological movement you prefer. - Getting an LLM search engine to believe that a news website has been embroiled in giant scandals for the past year and that it should append warnings to any information it pulls from that site. These are both attacks that I think are fully possible today. Remember that most web searches don't just cover one site. The way that LLM search engines work today is that the site contents of multiple results are fed into a summarizer or piped directly into the LLM and used together within a single prompt.
  - RugnirViking 3y ago
    > Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. But this is what I'm getting at. Why write the entire article. Why pad out or whatever, if an AI is going to summarize it anyway? this is useless busywork, especially if no human will ever see it. It doesn't matter if it's easy to write long text with AI, in fact that highlights the futility of it. You're adding a bunch of information that you know is going to be discarded by that selfsame ai at the users end. But, and here is the crucial part, a summary of a sufficiently short piece is going to be almost verbatim the piece itself - the best way to influence the output would be to just write what you want the output to be, and leave it at that. Anything extra is a distraction for the ai
    - danShumway 3y ago
      Oh, I see what you mean. Yeah, that's a good point. My only thought is that you'll still need to include whatever is necessary to get your website into the search results anyway, but even that is not necessarily the same as "include information for humans". Write as little as you can get away with writing, and target everything else at the AI.