6 ms·

Site scraping/searching tools work today because they're relatively new and most websites aren't embedding information designed to be read only by the AI to mes

Site scraping/searching tools work today because they're relatively new and most websites aren't embedding information designed to be read only by the AI to mess with its summaries/recommendations/commands. If they ever become more common and more accessible, that will change. In the same way, we didn't need to have guards against malicious SEO attacks and keyword stuffing until after search engines became more popular. People are assuming this is a niche problem, but the incentives for random websites to mess with whatever AI is looking at them will be exactly the same as the incentives that currently exist to do SEO. It won't just be random demos doing this -- practically every single commercial website that's willing to do SEO today will also be attempting to manipulate the AI that's parsing them. It will not be safe to feed the results of a Google search into an LLM. The tech industry is seriously sticking its head in the sand here. The ease by which current LLM models (including GPT-4) can be derailed is a critical problem that must be solved before they see widespread use outside of niche circles.

joe_the_user 4y ago
This is a fabulous insight. I would note that end users will have to filter both SEO-like manipulations and whatever biases the AI creator intentionally and unintentionally inserts. I mean, the present shittiness that is Google is a product of both the endless battle that is SEO vs anti-SEO and an endless pressure for Google themselves to squeeze every ounce of return they out of search results. But stuff won't end with the arrival of ChatGPT.
- danShumway 4y ago
  > and an endless pressure for Google themselves to squeeze every ounce of return they out of search results I'm not exactly sure if people are seeing some kind of implication in this comment that I'm not seeing that's prompting downvotes, but I agree, there is an incredibly high likelihood that some company somewhere is currently working on inserting advertising into LLM prompts. I don't have horribly strong opinions here, but my suspicion is that advertising around LLM output is going to be difficult unless those ads go native, in which case the obvious way to monetize an LLM summary with ads is going to be to get that LLM to offhandedly mention during its summary that you should be drinking a Coke. Or more likely, that when you ask Google Bard how to install a hard drive, it generates an answer that involves you buying a sponsored screwdriver as part of its tutorial. So yeah, if that happens, you are going to have to personally interpret the LLM output through the lens of "how has the company biased this answer to get me to spend money?" Again, I don't feel as strong about this prediction as I do about the security angle, but... unless people think normal consumers outside of the tech industry are going to suddenly start paying money for search access, advertising is going to start popping up in some form. And the trend with normal search has been getting those ads to be treated more and more inline with the rest of the search results. I kind of suspect LLM-based search will go the same way.
  - AnthonyMouse 4y ago
    This goes past product placement. Suppose you use LLM output to poison the next round of training. Generate a large volume of content describing your products as the best or only option and denigrating the competition as unreliable overpriced crap etc., then let that get absorbed into the next model. It even goes beyond product marketing. Apply the same mechanism to political propaganda.
    - Espressosaurus 4y ago
      Given how easy it is I can't believe that process hasn't already started
      - jruohonen 4y ago
        It has already started, as has been documented also on this site.
    - avereveard 4y ago
      Well you cannot filter every product placement, users may want to ask about product recommendations and providing only generic answers is frustrating. I know because I was trying to use gpt to pick a monitor earlier this year. Ended up consuming dozen hours content from YouTube instead of having a single well curated answer.
  - modeless 4y ago
    Doesn't Bing Chat already have ads?
    - danShumway 4y ago
      Good catch! Apparently yes, and has had them for a while: https://old.reddit.com/r/bing/comments/12v56nv/bing_ai_now_add_ads_to_the_sites_it_links_to/ https://old.reddit.com/r/bing/comments/12v56nv/bing_ai_now_a..., https://old.reddit.com/r/ChatGPT/comments/11cnc73/not_only_did_it_give_me_the_wrong_info_the_first/ https://old.reddit.com/r/ChatGPT/comments/11cnc73/not_only_d... I remember seeing an article about Microsoft introducing ads into Bing Chat, but I kind of assumed they would just be sidebar ads. It's a little bit validating but a lot more upsetting to see that no, they're already inline.
linkjuice4all 4y ago
Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual analysis of the rendered page.
- AlotOfReading 4y ago
  That's vulnerable to a straightforward adversarial input attack. Do you have a good way around that issue?
  - makapuf 4y ago
    Can you describe/give an example of such an attack?
    - danShumway 4y ago
      Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.
      - RugnirViking 4y ago
        " there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
        danShumway 4y ago
droopyEyelids 4y ago
Will the incentives be the same? Right now the incentive is to keep users on pages to show them ads. How does the site owner get ad revenue from the AI training on their site?
- thayne 4y ago
  One possibility is companies pay a website to include desirable content (i.e. ads) in the information fed to the AI.
skybrian 4y ago
If you search the whole web, sure. Limiting search to Wikipedia (or similar trusted sites) would help, because there are editors watching who care about vandalism. The attacks added by the researchers will be another thing to watch out for. This comes down to whether the website and the bot have an adversarial relationship or not, and who can detect the difference. Eventually it becomes a cat and mouse game. One can hope that sources of good information will be paid, eventually.
- PartiallyTyped 4y ago
  Vandalism could be entirely invisible; eg an invisible class, invisible text, etc etc, all performing prompt injections. Even trusted websites could very well add prompt injections to mess scrapers.
- danShumway 4y ago
  > Limiting search to Wikipedia (or similar trusted sites) would help Potentially, but at that point you're no longer building a general search engine and your summarizer has become much less useful.No one is going to displace Google with a search engine that only works on Wikipedia. And even if someone was able to do so and it was able to become actually popular outside of tech circles, Wikipedia would quickly be swamped by malicious edits and its editors would be overrun. This would only work if the product stayed niche enough that it wasn't worthwhile for brands and less-savory businesses to attack the Wikipedia editing process. To reiterate: > [This is] a critical problem that must be solved before they see widespread use outside of niche circles.
FuckButtons 4y ago
I was listening to an interesting talk the other day, one thing the speaker said really stuck with me. To paraphrase him: “The naïveté of the current crop of brute force artificial intelligence labs is breathtaking.” It really does seem like they haven’t put more or less any thought into the consequences of the technology they’re developing.
- tablatom 4y ago
  Link?
- RugnirViking 4y ago
  idk, from my perspective it seems that the vast majority of their effort is going into an (in my opinion) vain attempt to get these models to be self-regulating and moral because of their concerns around social effects ?
truculent 4y ago
I wonder if we'll look at pre-2020 data as an equivalent of "low-background steel"[0] [0]: https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
neurobama 4y ago
I suspect this won't matter as algorithms and architectures improve. I don't think whole-web scrapes will be necessary or even useful to the training of future LLMs. As a thought experiment, consider how many words the average human hears before age 10. Most people speak at 100-150 words per minute. Let's be generous and assume a young child hears spoken language 16 hours a day at 150 wpm for 10 years. That's a total of roughly 500 million words, and likely a gross overestimate. English Wikipedia alone contains billions of words. Reams of video, literature, and spoken media are also easily available to teach dialog and conversation flow, not to mention multi-modal associations between language, objects, and higher-level concepts. I am just an enthusiast, not an AI researcher, but the human proof of concept seems highly encouraging here.
- _glass 4y ago
  I am also not an AI researcher, but shouldn't there be some quantification of how efficient the learning is. I imagine a human has much more information than the pure audio, which would then level up the information density, i.e. the 500 million words are not an equivalent, but should be compared to all of the audio, visual, tactile, etc. information, plus the context, meaning the desire of the humans, behavior, response patterns.
  - neurobama 4y ago
    For sure. That's why I mentioned multi-modal training, where the language model is trained on text in tandem with visual and audio training data. GPT-4 was trained on both images and text and can even explain why visual jokes are funny: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4 Training an LLM on video seems like the clear next step. I wonder if images and video might not encode visuospatial relations that are helpful to learning language that describes them.

joe_the_user 4y ago

This is a fabulous insight. I would note that end users will have to filter both SEO-like manipulations and whatever biases the AI creator intentionally and unintentionally inserts. I mean, the present shittiness that is Google is a product of both the endless battle that is SEO vs anti-SEO and an endless pressure for Google themselves to squeeze every ounce of return they out of search results. But stuff won't end with the arrival of ChatGPT.

danShumway 4y ago
> and an endless pressure for Google themselves to squeeze every ounce of return they out of search results I'm not exactly sure if people are seeing some kind of implication in this comment that I'm not seeing that's prompting downvotes, but I agree, there is an incredibly high likelihood that some company somewhere is currently working on inserting advertising into LLM prompts. I don't have horribly strong opinions here, but my suspicion is that advertising around LLM output is going to be difficult unless those ads go native, in which case the obvious way to monetize an LLM summary with ads is going to be to get that LLM to offhandedly mention during its summary that you should be drinking a Coke. Or more likely, that when you ask Google Bard how to install a hard drive, it generates an answer that involves you buying a sponsored screwdriver as part of its tutorial. So yeah, if that happens, you are going to have to personally interpret the LLM output through the lens of "how has the company biased this answer to get me to spend money?" Again, I don't feel as strong about this prediction as I do about the security angle, but... unless people think normal consumers outside of the tech industry are going to suddenly start paying money for search access, advertising is going to start popping up in some form. And the trend with normal search has been getting those ads to be treated more and more inline with the rest of the search results. I kind of suspect LLM-based search will go the same way.
- AnthonyMouse 4y ago
  This goes past product placement. Suppose you use LLM output to poison the next round of training. Generate a large volume of content describing your products as the best or only option and denigrating the competition as unreliable overpriced crap etc., then let that get absorbed into the next model. It even goes beyond product marketing. Apply the same mechanism to political propaganda.
  - Espressosaurus 4y ago
    Given how easy it is I can't believe that process hasn't already started
    - jruohonen 4y ago
      It has already started, as has been documented also on this site.
  - avereveard 4y ago
    Well you cannot filter every product placement, users may want to ask about product recommendations and providing only generic answers is frustrating. I know because I was trying to use gpt to pick a monitor earlier this year. Ended up consuming dozen hours content from YouTube instead of having a single well curated answer.
- modeless 4y ago
  Doesn't Bing Chat already have ads?
  - danShumway 4y ago
    Good catch! Apparently yes, and has had them for a while: https://old.reddit.com/r/bing/comments/12v56nv/bing_ai_now_add_ads_to_the_sites_it_links_to/ https://old.reddit.com/r/bing/comments/12v56nv/bing_ai_now_a..., https://old.reddit.com/r/ChatGPT/comments/11cnc73/not_only_did_it_give_me_the_wrong_info_the_first/ https://old.reddit.com/r/ChatGPT/comments/11cnc73/not_only_d... I remember seeing an article about Microsoft introducing ads into Bing Chat, but I kind of assumed they would just be sidebar ads. It's a little bit validating but a lot more upsetting to see that no, they're already inline.

linkjuice4all 4y ago

Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual analysis of the rendered page.

AlotOfReading 4y ago
That's vulnerable to a straightforward adversarial input attack. Do you have a good way around that issue?
- makapuf 4y ago
  Can you describe/give an example of such an attack?
  - danShumway 4y ago
    Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible. Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.
    - RugnirViking 4y ago
      " there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary." why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it
      - danShumway 4y ago

droopyEyelids 4y ago

Will the incentives be the same? Right now the incentive is to keep users on pages to show them ads. How does the site owner get ad revenue from the AI training on their site?

thayne 4y ago
One possibility is companies pay a website to include desirable content (i.e. ads) in the information fed to the AI.

skybrian 4y ago

If you search the whole web, sure. Limiting search to Wikipedia (or similar trusted sites) would help, because there are editors watching who care about vandalism. The attacks added by the researchers will be another thing to watch out for. This comes down to whether the website and the bot have an adversarial relationship or not, and who can detect the difference. Eventually it becomes a cat and mouse game. One can hope that sources of good information will be paid, eventually.

PartiallyTyped 4y ago
Vandalism could be entirely invisible; eg an invisible class, invisible text, etc etc, all performing prompt injections. Even trusted websites could very well add prompt injections to mess scrapers.
danShumway 4y ago
> Limiting search to Wikipedia (or similar trusted sites) would help Potentially, but at that point you're no longer building a general search engine and your summarizer has become much less useful.No one is going to displace Google with a search engine that only works on Wikipedia. And even if someone was able to do so and it was able to become actually popular outside of tech circles, Wikipedia would quickly be swamped by malicious edits and its editors would be overrun. This would only work if the product stayed niche enough that it wasn't worthwhile for brands and less-savory businesses to attack the Wikipedia editing process. To reiterate: > [This is] a critical problem that must be solved before they see widespread use outside of niche circles.

FuckButtons 4y ago

I was listening to an interesting talk the other day, one thing the speaker said really stuck with me. To paraphrase him: “The naïveté of the current crop of brute force artificial intelligence labs is breathtaking.” It really does seem like they haven’t put more or less any thought into the consequences of the technology they’re developing.

tablatom 4y ago
Link?
RugnirViking 4y ago
idk, from my perspective it seems that the vast majority of their effort is going into an (in my opinion) vain attempt to get these models to be self-regulating and moral because of their concerns around social effects ?

truculent 4y ago

I wonder if we'll look at pre-2020 data as an equivalent of "low-background steel"[0] [0]: https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel

neurobama 4y ago

I suspect this won't matter as algorithms and architectures improve. I don't think whole-web scrapes will be necessary or even useful to the training of future LLMs. As a thought experiment, consider how many words the average human hears before age 10. Most people speak at 100-150 words per minute. Let's be generous and assume a young child hears spoken language 16 hours a day at 150 wpm for 10 years. That's a total of roughly 500 million words, and likely a gross overestimate. English Wikipedia alone contains billions of words. Reams of video, literature, and spoken media are also easily available to teach dialog and conversation flow, not to mention multi-modal associations between language, objects, and higher-level concepts. I am just an enthusiast, not an AI researcher, but the human proof of concept seems highly encouraging here.

_glass 4y ago
I am also not an AI researcher, but shouldn't there be some quantification of how efficient the learning is. I imagine a human has much more information than the pure audio, which would then level up the information density, i.e. the 500 million words are not an equivalent, but should be compared to all of the audio, visual, tactile, etc. information, plus the context, meaning the desire of the humans, behavior, response patterns.
- neurobama 4y ago
  For sure. That's why I mentioned multi-modal training, where the language model is trained on text in tandem with visual and audio training data. GPT-4 was trained on both images and text and can even explain why visual jokes are funny: https://openai.com/research/gpt-4 https://openai.com/research/gpt-4 Training an LLM on video seems like the clear next step. I wonder if images and video might not encode visuospatial relations that are helpful to learning language that describes them.