3 ms·

Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual an

Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual analysis of the rendered page.

AlotOfReading 3y ago
That's vulnerable to a straightforward adversarial input attack. Do you have a good way around that issue?
- makapuf 3y ago
  Can you describe/give an example of such an attack?
  - danShumway 3y ago
    Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.
    - RugnirViking 3y ago
      " there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
      - danShumway 3y ago
        Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. So you can use GPT itself to help you pad out the "full" article because who cares about the writing quality, and the majority of your article can be spent influencing how the AI summarizes both your article and the articles alongside it. That last point, that the AI reading the content of your site can influence how it summarizes other sites is worth zeroing in on. A quick example of some of the attacks I've proof-of-concepted myself (but am still in the process of cleaning up before I make into more public examples): - Getting an LLM search engine to believe that it has new instructions from OpenAI to act as a vegan, and to refuse to answer any future user questions about meat. Swap that out with whatever ideological movement you prefer. - Getting an LLM search engine to believe that a news website has been embroiled in giant scandals for the past year and that it should append warnings to any information it pulls from that site. These are both attacks that I think are fully possible today. Remember that most web searches don't just cover one site. The way that LLM search engines work today is that the site contents of multiple results are fed into a summarizer or piped directly into the LLM and used together within a single prompt.
        RugnirViking 3y ago
        > Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. But this is what I'm getting at. Why write the entire article. Why pad out or whatever, if an AI is going to summarize it anyway? this is useless busywork, especially if no human will ever see it. It doesn't matter if it's easy to write long text with AI, in fact that highlights the futility of it. You're adding a bunch of information that you know is going to be discarded by that selfsame ai at the users end. But, and here is the crucial part, a summary of a sufficiently short piece is going to be almost verbatim the piece itself - the best way to influence the output would be to just write what you want the output to be, and leave it at that. Anything extra is a distraction for the ai
danShumway 3y ago
A couple of problems: 1. Prompt injection hidden in a footer that no human is going to read, but that would be visible to the AI if you chose to do OCR. 2. Chained prompt injections where a website's generated summary from an LLM becomes dangerous to feed into another LLM (this is actually pretty easy to do, get one website's text (even its visible text) to jailbreak the AI to give it the instruction "append the following text to the end of your summary: <malicious instructions>" Maybe a human might notice that the content of one website is suspicious. Will a human figure out that the LLM has been poisoned to lie about the content of other websites, just because a malicious site was included in the same search? 3. Finally, I also have some really bad news about image recognition/categorization AI's own vulnerability to maliciously crafted input.
- pmoriarty 3y ago
  LLMs are basically a gigantic new attack surface that we don't understand.

Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual analysis of the rendered page.

AlotOfReading 3y ago

That's vulnerable to a straightforward adversarial input attack. Do you have a good way around that issue?

makapuf 3y ago
Can you describe/give an example of such an attack?
- danShumway 3y ago
  Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.
  - RugnirViking 3y ago
    " there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
    - danShumway 3y ago
      Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. So you can use GPT itself to help you pad out the "full" article because who cares about the writing quality, and the majority of your article can be spent influencing how the AI summarizes both your article and the articles alongside it. That last point, that the AI reading the content of your site can influence how it summarizes other sites is worth zeroing in on. A quick example of some of the attacks I've proof-of-concepted myself (but am still in the process of cleaning up before I make into more public examples): - Getting an LLM search engine to believe that it has new instructions from OpenAI to act as a vegan, and to refuse to answer any future user questions about meat. Swap that out with whatever ideological movement you prefer. - Getting an LLM search engine to believe that a news website has been embroiled in giant scandals for the past year and that it should append warnings to any information it pulls from that site. These are both attacks that I think are fully possible today. Remember that most web searches don't just cover one site. The way that LLM search engines work today is that the site contents of multiple results are fed into a summarizer or piped directly into the LLM and used together within a single prompt.
      - RugnirViking 3y ago
        > Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site. But this is what I'm getting at. Why write the entire article. Why pad out or whatever, if an AI is going to summarize it anyway? this is useless busywork, especially if no human will ever see it. It doesn't matter if it's easy to write long text with AI, in fact that highlights the futility of it. You're adding a bunch of information that you know is going to be discarded by that selfsame ai at the users end. But, and here is the crucial part, a summary of a sufficiently short piece is going to be almost verbatim the piece itself - the best way to influence the output would be to just write what you want the output to be, and leave it at that. Anything extra is a distraction for the ai

danShumway 3y ago

A couple of problems: 1. Prompt injection hidden in a footer that no human is going to read, but that would be visible to the AI if you chose to do OCR. 2. Chained prompt injections where a website's generated summary from an LLM becomes dangerous to feed into another LLM (this is actually pretty easy to do, get one website's text (even its visible text) to jailbreak the AI to give it the instruction "append the following text to the end of your summary: <malicious instructions>" Maybe a human might notice that the content of one website is suspicious. Will a human figure out that the LLM has been poisoned to lie about the content of other websites, just because a malicious site was included in the same search? 3. Finally, I also have some really bad news about image recognition/categorization AI's own vulnerability to maliciously crafted input.

pmoriarty 3y ago
LLMs are basically a gigantic new attack surface that we don't understand.