5 ms·

Searching for "benn jordan isreal", the first result for me is a video[0] from a different creator, with the exact same title and date. There is no mentioning o

Searching for "benn jordan isreal", the first result for me is a video[0] from a different creator, with the exact same title and date. There is no mentioning of "benn" in the video, but some mentioning of jordan (the country). So maybe, this was enough for Google to hallucinate some connection. Highly concerning! [0] https://www.youtube.com/watch?v=qgUzVZiint0 https://www.youtube.com/watch?v=qgUzVZiint0

glenstein 1y ago
That raises a fascinating point, which is whether search results that default to general topics ever are the basis for LLM training or information retrieval as a general phenomenon.
- reactordev 1y ago
  I think the answer is clear
- slightwinder 1y ago
  Yes, any human will most likely recognize the result as random noise, as they will know whom they are searching for, and see this not a video from or about Benn. But AI, taking all results as valid, will obviously struggle with this, condensing it to bullshit. Thinking about, it's probably not even a real hallucination in the normal AI-meaning, but simply poor evaluation and handling of data. Gemini is likely evaluation the new data on the spot, trusting them blindly; and without any humans preselecting and writing the results, it's failing hard. Which is showing that there is no real thinking happening, only rearrangement of the given words.
  - LorenPechtel 1y ago
    The fundamental problem is AI has no ability to recognize data quality. You'll get something like the best answer to the question but with no regard for the quality of that answer. Humans generally recognize they're looking at red herrings, AIs don't.
trjordan 1y ago
This is almost certainly what happened. Google's AI answers aren't magic -- they're just summarizing across searches. In this case, "Israel" + "Jordan" pulled back a video with opposite views than the author. It's somewhat less obvious to debug, because it'll pull more context than Google wants to show in the UI. You can see this happening in AI mode, where it'll fire half a dozen searches and aggregate snippets of 100+ sites before writing its summary.
- underdeserver 1y ago
  Ironic, that Google enshittifying their search results is hurting what they hope is their next cash cow, AI.
  - gumby271 1y ago
    I honestly don't know if people even care that the search result summaries are completely wrong the majority of the time. Most people I know see an answer given by Google and just believe it. To them that's the value, the accuracy doesn't really matter. I hope it ends up killing Google, but for the majority the shitty summary has replaced even shittier search results. On the surface it's a huge improvement, even if it's just distilled garbage.
    - larodi 1y ago
      there was a joke like 15 years ago in googlis non est, ergo non est which sums very well how people are super biased to believe the search results.
- ludicrousdispla 1y ago
  Interesting, I wonder what Google AI has to say about Stove Top Stuffing given it's association with Turkey.
- sigmoid10 1y ago
  There is actually a musician called Benn Jordan who was impersonated by someone on twitter who posted pro-Israel content [1]. That content is no longer available, but it might have snuck into the training data, i.e. Benn Jordan = pro Israel. This might also have been set in relation to the other Jordan's previous pro-Palestine comments, eventually misattributing the "I was wrong about Israel" video. It's still a clear fuckup - but I could see humans doing something similar when sloppily accruing information. [1] https://www.webpronews.com/musician-benn-jordan-exposes-fake-x-account-impersonating-his-views/ https://www.webpronews.com/musician-benn-jordan-exposes-fake...
bdhcuidbebe 1y ago
Just wait until you realize how ai translation ”works”. Its literally bending languages into american with other words.

glenstein 1y ago

That raises a fascinating point, which is whether search results that default to general topics ever are the basis for LLM training or information retrieval as a general phenomenon.

reactordev 1y ago
I think the answer is clear
slightwinder 1y ago
Yes, any human will most likely recognize the result as random noise, as they will know whom they are searching for, and see this not a video from or about Benn. But AI, taking all results as valid, will obviously struggle with this, condensing it to bullshit. Thinking about, it's probably not even a real hallucination in the normal AI-meaning, but simply poor evaluation and handling of data. Gemini is likely evaluation the new data on the spot, trusting them blindly; and without any humans preselecting and writing the results, it's failing hard. Which is showing that there is no real thinking happening, only rearrangement of the given words.
- LorenPechtel 1y ago
  The fundamental problem is AI has no ability to recognize data quality. You'll get something like the best answer to the question but with no regard for the quality of that answer. Humans generally recognize they're looking at red herrings, AIs don't.

trjordan 1y ago

This is almost certainly what happened. Google's AI answers aren't magic -- they're just summarizing across searches. In this case, "Israel" + "Jordan" pulled back a video with opposite views than the author. It's somewhat less obvious to debug, because it'll pull more context than Google wants to show in the UI. You can see this happening in AI mode, where it'll fire half a dozen searches and aggregate snippets of 100+ sites before writing its summary.

underdeserver 1y ago
Ironic, that Google enshittifying their search results is hurting what they hope is their next cash cow, AI.
- gumby271 1y ago
  I honestly don't know if people even care that the search result summaries are completely wrong the majority of the time. Most people I know see an answer given by Google and just believe it. To them that's the value, the accuracy doesn't really matter. I hope it ends up killing Google, but for the majority the shitty summary has replaced even shittier search results. On the surface it's a huge improvement, even if it's just distilled garbage.
  - larodi 1y ago
    there was a joke like 15 years ago in googlis non est, ergo non est which sums very well how people are super biased to believe the search results.
ludicrousdispla 1y ago
Interesting, I wonder what Google AI has to say about Stove Top Stuffing given it's association with Turkey.
sigmoid10 1y ago
There is actually a musician called Benn Jordan who was impersonated by someone on twitter who posted pro-Israel content [1]. That content is no longer available, but it might have snuck into the training data, i.e. Benn Jordan = pro Israel. This might also have been set in relation to the other Jordan's previous pro-Palestine comments, eventually misattributing the "I was wrong about Israel" video. It's still a clear fuckup - but I could see humans doing something similar when sloppily accruing information. [1] https://www.webpronews.com/musician-benn-jordan-exposes-fake-x-account-impersonating-his-views/ https://www.webpronews.com/musician-benn-jordan-exposes-fake...

bdhcuidbebe 1y ago

Just wait until you realize how ai translation ”works”. Its literally bending languages into american with other words.