4 ms·

The AI Overviews outrage is only the beginning. It has been 11 days since they made it public in the US, and it had been in experimentation for over a year befo

The AI Overviews outrage is only the beginning. It has been 11 days since they made it public in the US, and it had been in experimentation for over a year before that. Already now, a lot of users will associate Google AI Overviews with nonsense; eat rocks, cook spaghetti with gasoline. Google also showed how easily these AI Overviews can be manipulated, since they use RAG. I think the Slopception is only going to get worse. Slop in. Slop out. And also, Google Research just happened to be sitting on AGREE[0]: > a learning-based framework that enables LLMs to provide accurate citations in their responses, making them more reliable and increasing user trust Which was published yesterday. I tried to submit it[1] but it didn’t get much love. [0]: https://research.google/blog/effective-large-language-model-adaptation-for-improved-grounding/ https://research.google/blog/effective-large-language-model-... [1]: https://news.ycombinator.com/item?id=40469518 https://news.ycombinator.com/item?id=40469518

tomrod 2y ago
<3 from me. We are in the rough and tumble time of new tech. As an analogy, trains didn't always have the distance between their tracks as uniform (called gauge). It strikes me that we are in a similar time of invention. If you have ever read old patents (my experience being older pop book like "Strange Stories, Amazing Facts" and random blogposts) you'll see the truly bizzare.
- shh_labs 2y ago
  But the trains with differently sized tracks didn’t fall off the tracks because (and I imagine there is some obscure exception to this, but for the very most part) someone had made sure that the systems were safe. The distinction here is that Google is rolling out half-baked rubbish that is palpably unreliable. No sane person is going to eat rocks or put glue on their pizza, but the race is on to find some genuinely dangerous answer from this system. It would be regrettable if the result of rolling out this type of product too soon has the effect of eroding trust in AI-based search, and delaying people from engaging with it when it actually is ready.
  - paulmd 2y ago
    > The distinction here is that Google is rolling out half-baked rubbish that is palpably unreliable. nowhere in that sentence did you mention AI, though. If you take AI out of the picture, isn’t it equally problematic if they had some other mechanism, say some kind of analytical/stochastic mechanism, that selected data retrieved from actual results, and cited those? Because, that’s what they’re doing. Google isn’t using an AI. They’re using a stochastic textual generator driven by results of the search. Sometimes it spits out weird and incorrect answers, especially when high-ranking sites suddenly spoof malicious results - that is a general problem with any attempt to pull data out of the results automatically, including the lesser variants of this approach they’ve deployed for decades. The problem is (a) google putting out a poor product, similar to the pre-google era of search engines being really bad, and (b) high-ranking sites engaging in malicious and disruptive behavior, for which they probably should be de-ranked or blacklisted. But you cannot gave this one both ways, it can’t be an “AI” when you want to be an maximalist and “just a textual stochastic algorithm” when you want to minimize and downplay. By the latter terms Google isn’t even using AI here, just a stochastic algorithm, driven by the search results. It’s just a bad one. So you’re arguing against an approach that doesn’t exist anyway. Again: why specifically is google answer problematic, given that it’s been rolled out for like 15+ years now? The new algorithm behind it just kinda sucks, but the feature isn’t new. Citing and summarizing results has been a thing for a long time, and naturally it’s not always correct/accurate. This isn’t the first time people have been upset over it either - newspapers got mad about this summarization so they got a law passed in Canada which outlawed it. https://www.reuters.com/technology/canada-news-industry-body-backs-googles-concerns-about-online-news-law-2023-10-12/ https://www.reuters.com/technology/canada-news-industry-body...
  - tomrod 2y ago
    The operation of the trains probably ran with precision, despite the need to switch between tracks and trains of a different gauge, and despite the occasional accident. But, like Google's garbage, the user experience for shipping or riding was abysmal.
viking123 2y ago
It's going to get much much worse with all types of media tbh.
- skilled 2y ago
  Scam callers are going to have a field day with audio for sure. Maybe I am not thinking about it right but a lot of this does feel surreal and I just can’t see the light at the end this tunnel. No regulation, rapid development, no protective measures even for the very basic attack vectors.
- fullshark 2y ago
  Maybe people will finally be willing to pay for it.
polski-g 2y ago
Google AI was so much better when they used it to do useful things like create the best starcraft player in the world. Imagine if strategy game companies could license/extend DeepMind game AI.
adolph 2y ago
Enjoy the nonsense, this is the “egg freckles”[0] stage of the technology.
raydev 2y ago
> Already now, a lot of users will associate Google AI Overviews with nonsense A lot of tech-minded folks who actively follow this stuff on Twitter et al make this association, but the average person was already defaulting to reading the first (likely incorrect) hit on Google and moving on with their day. I'm not sure they'll notice a dip in quality until it directly contradicts something they know about confidently.

tomrod 2y ago

<3 from me. We are in the rough and tumble time of new tech. As an analogy, trains didn't always have the distance between their tracks as uniform (called gauge). It strikes me that we are in a similar time of invention. If you have ever read old patents (my experience being older pop book like "Strange Stories, Amazing Facts" and random blogposts) you'll see the truly bizzare.

shh_labs 2y ago
But the trains with differently sized tracks didn’t fall off the tracks because (and I imagine there is some obscure exception to this, but for the very most part) someone had made sure that the systems were safe. The distinction here is that Google is rolling out half-baked rubbish that is palpably unreliable. No sane person is going to eat rocks or put glue on their pizza, but the race is on to find some genuinely dangerous answer from this system. It would be regrettable if the result of rolling out this type of product too soon has the effect of eroding trust in AI-based search, and delaying people from engaging with it when it actually is ready.
- paulmd 2y ago
  > The distinction here is that Google is rolling out half-baked rubbish that is palpably unreliable. nowhere in that sentence did you mention AI, though. If you take AI out of the picture, isn’t it equally problematic if they had some other mechanism, say some kind of analytical/stochastic mechanism, that selected data retrieved from actual results, and cited those? Because, that’s what they’re doing. Google isn’t using an AI. They’re using a stochastic textual generator driven by results of the search. Sometimes it spits out weird and incorrect answers, especially when high-ranking sites suddenly spoof malicious results - that is a general problem with any attempt to pull data out of the results automatically, including the lesser variants of this approach they’ve deployed for decades. The problem is (a) google putting out a poor product, similar to the pre-google era of search engines being really bad, and (b) high-ranking sites engaging in malicious and disruptive behavior, for which they probably should be de-ranked or blacklisted. But you cannot gave this one both ways, it can’t be an “AI” when you want to be an maximalist and “just a textual stochastic algorithm” when you want to minimize and downplay. By the latter terms Google isn’t even using AI here, just a stochastic algorithm, driven by the search results. It’s just a bad one. So you’re arguing against an approach that doesn’t exist anyway. Again: why specifically is google answer problematic, given that it’s been rolled out for like 15+ years now? The new algorithm behind it just kinda sucks, but the feature isn’t new. Citing and summarizing results has been a thing for a long time, and naturally it’s not always correct/accurate. This isn’t the first time people have been upset over it either - newspapers got mad about this summarization so they got a law passed in Canada which outlawed it. https://www.reuters.com/technology/canada-news-industry-body-backs-googles-concerns-about-online-news-law-2023-10-12/ https://www.reuters.com/technology/canada-news-industry-body...
- tomrod 2y ago
  The operation of the trains probably ran with precision, despite the need to switch between tracks and trains of a different gauge, and despite the occasional accident. But, like Google's garbage, the user experience for shipping or riding was abysmal.

viking123 2y ago

It's going to get much much worse with all types of media tbh.

skilled 2y ago
Scam callers are going to have a field day with audio for sure. Maybe I am not thinking about it right but a lot of this does feel surreal and I just can’t see the light at the end this tunnel. No regulation, rapid development, no protective measures even for the very basic attack vectors.
fullshark 2y ago
Maybe people will finally be willing to pay for it.

polski-g 2y ago

Google AI was so much better when they used it to do useful things like create the best starcraft player in the world. Imagine if strategy game companies could license/extend DeepMind game AI.

adolph 2y ago

Enjoy the nonsense, this is the “egg freckles”[0] stage of the technology.

raydev 2y ago

> Already now, a lot of users will associate Google AI Overviews with nonsense A lot of tech-minded folks who actively follow this stuff on Twitter et al make this association, but the average person was already defaulting to reading the first (likely incorrect) hit on Google and moving on with their day. I'm not sure they'll notice a dip in quality until it directly contradicts something they know about confidently.