4 ms·

You’ve seen people game adsense It’s gunna be even wilder when people realise they have an incentive to seed fake information on the internet to game AI produc

You’ve seen people game adsense It’s gunna be even wilder when people realise they have an incentive to seed fake information on the internet to game AI product recommendations I’ve already bought stuff based off of an AI suggestion, I didn’t even consider it would be so easy to influence the suggestion. Just two research papers? Mad.

ccgreg 6mo ago
That's already been happening for more than a year now.
r721 6mo ago
This has a name already: "AEO (Answer Engine Optimization)".
- vrganj 6mo ago
  I hate people. Things could be so good if we weren't the way we are.
baobun 6mo ago
All it takes to become world champion is a blog. https://www.bbc.com/future/article/20260218-i-hacked-chatgpt-and-googles-ai-and-it-only-took-20-minutes https://www.bbc.com/future/article/20260218-i-hacked-chatgpt...
simianwords 6mo ago
You are pointing at something that is orthogonal to this paper. The LLM did not randomly recommend or bring this disease up to people - it merely assumed the disease was true when the preprint was pointed at it.
- simmerup 6mo ago
  The LLM bought up the disease because some person put a fake journal in its training data. If the person put their product as th definitive cure for the made up disease, the LLM probably would have mentioned that too. > merely assumed the disease was true when the preprint was pointed at it. What do you mean by preprint pointed at it? It being the disease?
  - simianwords 6mo ago
    > The LLM bought up the disease because some person put a fake journal in its training data. This is not true - the model was not trained on this fake disease. It brought it up because it found it during real time search. >What do you mean by preprint pointed at it? It being the disease? On this I'm wrong - it turned out that the model brought up this disease even when not mentioning it explicitly.
    - simmerup 6mo ago
      Interesting, will be looking into RAG now. I assume Claude was retrained regularly but Opus for example was last trained August 2025. Way older than I thought it’d be
stingraycharles 6mo ago
This is already a thing for a year or so, SEO for AI results to make sure that your products are recommended in ChatGPT. https://citeworksstudio.com/ https://citeworksstudio.com/ is a decent one.
scoofy 6mo ago
It’s not mad… it’s the same damn thing as taking Wikipedia articles as Truth without looking at the citations and verifying them. AI research is for research, not for blindly accepting. If you’re looking for Truth you need to institute a gatekeeper that does that homework for you.

ccgreg 6mo ago

That's already been happening for more than a year now.

r721 6mo ago

This has a name already: "AEO (Answer Engine Optimization)".

vrganj 6mo ago
I hate people. Things could be so good if we weren't the way we are.

baobun 6mo ago

All it takes to become world champion is a blog. https://www.bbc.com/future/article/20260218-i-hacked-chatgpt-and-googles-ai-and-it-only-took-20-minutes https://www.bbc.com/future/article/20260218-i-hacked-chatgpt...

simianwords 6mo ago

You are pointing at something that is orthogonal to this paper. The LLM did not randomly recommend or bring this disease up to people - it merely assumed the disease was true when the preprint was pointed at it.

simmerup 6mo ago
The LLM bought up the disease because some person put a fake journal in its training data. If the person put their product as th definitive cure for the made up disease, the LLM probably would have mentioned that too. > merely assumed the disease was true when the preprint was pointed at it. What do you mean by preprint pointed at it? It being the disease?
- simianwords 6mo ago
  > The LLM bought up the disease because some person put a fake journal in its training data. This is not true - the model was not trained on this fake disease. It brought it up because it found it during real time search. >What do you mean by preprint pointed at it? It being the disease? On this I'm wrong - it turned out that the model brought up this disease even when not mentioning it explicitly.
  - simmerup 6mo ago
    Interesting, will be looking into RAG now. I assume Claude was retrained regularly but Opus for example was last trained August 2025. Way older than I thought it’d be

stingraycharles 6mo ago

This is already a thing for a year or so, SEO for AI results to make sure that your products are recommended in ChatGPT. https://citeworksstudio.com/ https://citeworksstudio.com/ is a decent one.

scoofy 6mo ago

It’s not mad… it’s the same damn thing as taking Wikipedia articles as Truth without looking at the citations and verifying them. AI research is for research, not for blindly accepting. If you’re looking for Truth you need to institute a gatekeeper that does that homework for you.