15 ms·
AI Overviews: About last week
- gmerc 2y agoThis is remarkable and so full of misdirection, it’s hard to know where to start? Is it the “lack of contemplation” invoked by a company enough data (see the recent google search leaks) to make the NSA blush? Is it the fact that it glances over the vast amount of data about this one article back from 2021 that all screams satire? Is it that google bought reddit data which is full of context about the topic and content in /r/theonion and /r/notheonion? Is it that when I tried 8 other LLMs, all the way down to phi3-llava, a 3.8B vision model on a text only screenshot of the first two paragraph it responded with > “ Berkeley did not actually recommend eating small rocks as part of a healthy diet. The idea of incorporating sediment into one's diet, let alone suggesting that people should aim for a daily intake of at least one small rock, is absurd and completely contradicts scientific consensus” as did every competing model in one form or another (gpt4, claude, llama3-7B, wizardlm), indicating that even an edge model as enough information in its weights to identify the article as satire even without google’s data warehouse. Is it the language talking about Gemini “faithfully complying” and “nobody asked the question”, which seems to paint the technology as a faithful hound let down by its human masters? Is it that Google says “it’s hard” as if they were not the best resourced tech company in the world fully in charge of their own roadmap and technology stack and the luxury to pay off tens of thousands of workers because AI is so awesome? Is it that Google chose to shove this down the throats of SERP customers rather launching a new product to use their monopoly to quash a better executing competiton? Is it that there a pattern of similar issues (“Opps, Asian and African American SS troops”)? Is it that it’s impossible to tell the excuses from a VP of Product from an LLM hallucination? (That may actually reinforce the economic potential of the technology)? Is it that we are apparently so gullible, we deserve being fed something like this? I almost feel bad for Perplexity because a lot of people will take away “the technology is a total failure” rather than “Google is so desperate they’d rather burn down the ecosystem and tell everyone to eat glue [1] than let them get 1% of search” from this [1] https://snekoil.omg.lol https://snekoil.omg.lol
- greyadept 2y agoTo detect the faked screenshot results, the post asks us to run the queries for ourselves and see what happens. But where does that leave people outside the US who don't have access to AI Overviews?
- xwolfi 2y agoBah, we just have to wait for the next batch of GPU they get so that giant datacenters can be equipped to give us 0.2% more accurate results about how to eat rocks.. Cant wait for the next-gen ads !
- Xenoamorphous 2y agoPlus they say they’ve already rolled out some changes, no?
- ADeerAppeared 2y agoThat's the point. The dirty secret of a lot of AI is that they rush out a hotfix for any such issues that pop up, and then just insist the issue was fake. "You can't replicate the issue, our AI is flawless, DO NOT LOOK AT THE PATCH ROLLOUT BEHIND THE CURTAIN".
- lol768 2y agoIt's also non-deterministic, right? So what does it prove if I get a different answer to someone on Twitter?
- fooker 2y agoWow, they are basically admitting they can’t solve the problem. Whoever solves this problem, hence has a good shot at being the next Google. OpenAI is probably not there yet, and it’s unclear if they will be. Apple? Some dark horse ?
- elevatedastalt 2y agoLiterally no one has solved it yet.
- fooker 2y agoI agree, kind of similar to ranked web search pre-pagerank.
- gmerc 2y agoLiterally unsolvable. No way even a 3.8B parameter vision model for the edge (phi3-llava) trained on the internet of hate could cluster the tokens of the first to paragraphs together with bullshit. https://cdn.some.pics/snekoil/66594bca93e19.jpg https://cdn.some.pics/snekoil/66594bca93e19.jpg No way llama3-7B Q4 would be able to under the same on a home GPU https://cdn.some.pics/snekoil/66594cb8cddcd.jpg https://cdn.some.pics/snekoil/66594cb8cddcd.jpg No way GPT4 can perform a web search to determine the source is the onion and flag it as satirical or summarize the page and flag it satirical https://cdn.some.pics/snekoil/66594e1a3de29.jpg https://cdn.some.pics/snekoil/66594e1a3de29.jpg No way Claude can flag that it’s satire. https://cdn.some.pics/snekoil/665955fb22618.jpg https://cdn.some.pics/snekoil/665955fb22618.jpg No way WizardLM30B would suggest it’s satire because eating rocks is not scientifically backed. Yes, it’s not reliable, but it shows that there’s enough information encoded in even small edge models without access to the largest data warehouse of web content and human interaction and hundreds of millions of dollars in content deals to put a “likely fake on it”. Yes, it may be unsolved at SERP scale, not chat scale, but Google chose to have this fight on SERP and launch so they have to take the hits. Yes replacing context with Gemini god quotes to monopolize traffic onto google is bad, but given the above results augmenting results rather than appropriating them seems to have some potential as seen with Perplexity. Yes LLMs are flawed, but as some competitors show they can improve on the very low bar that is Google search results and not just by removing the paid misinformation / ads. Google showed they can make it worse.
- soulofmischief 2y agoA masterclass in gaslighting users. Google just can't take the L on this one. I don't know anyone personally who found the AI overviews to be helpful. For one, searching is an act of research, corroborating multiple sources, and the current implementation of AI overviews obstructs this process. There is no trust for these services yet, and for good reason; if the AI is even 5% likely to return a harmful or "satirical" result, it should be treated as a completely unreliable source, and the user should fall back to reading the provided sources anyway. So I'm not sure how this is more helpful than regular vector search. Just about the only useful thing about AI overviews is that it places content at the top of the page, directly beneath the search bar, instead of three full pages down, below two awfully-designed widgets and a dozen sponsored, often nefarious results. That was nice. Too bad it's just another useless widget. Does anyone left at Google with power actually care about delivering a good search product? Earnest question.
- gmerc 2y agoWow so much google brigading going on. this does not deserve the downvotes it gets.
- jsemrau 2y ago"We hold ourselves to a high standard, as do our users, so we expect and appreciate the feedback, and take it seriously." In terms of AI Safety, I believe Anthrophic has built the right culture and mindset. In my opinion there is a difference between doing right and say that you are doing right. Usually if you do the latter you actually don't. It feels like BigTech companies are scrambling towards not falling behind in the AI race, move fast and break things, with little consideration of the potential harmful side affects of edge use cases for covert influence and similar.
- gmerc 2y agoit’s unfair but hey, Google chose to compete. https://cdn.some.pics/snekoil/665955fb22618.jpg https://cdn.some.pics/snekoil/665955fb22618.jpg
- programjames 2y agoI'm usually pretty cynical about these kinds of posts, but Google seems to be pretty honest here. It's a pretty reasonable take on what went wrong and what they're working on to fix it. I still don't particularly like the AI Overview though.
- gmerc 2y agoExcuse me, lack of information is a misdirect. They have data warehouse of information that knows where this article is coming from, and every fucking url ever put into chrome. They bought the rights to /r/the_onion It’s an onion article. Directly from the page. Are they saying “the mighty google search” + “AI” is going to be somehow worse because they don’t have enough data about the fucking Onion article with dozens of hits in 2021? Even a 3.8B parameter edge model (phi3-mini) has enough satire token clustered around the first two paragraphs to flag it accordingly. https://cdn.some.pics/snekoil/66594bca93e19.jpg https://cdn.some.pics/snekoil/66594bca93e19.jpg as does llama3-7B https://cdn.some.pics/snekoil/66594cb8cddcd.jpg https://cdn.some.pics/snekoil/66594cb8cddcd.jpg as does GPT4 https://cdn.some.pics/snekoil/66594e1a3de29.jpg https://cdn.some.pics/snekoil/66594e1a3de29.jpg as does Claude https://cdn.some.pics/snekoil/665955fb22618.jpg https://cdn.some.pics/snekoil/665955fb22618.jpg Instead they gaslight us with words like “faithful” and “nobody asked the question”. Their entire competition executes better. Just because they AI wash the search data to boost their stock narrative we don’t have to accept their lame limitations narrative. They chose this fight. Also see my longer commentary at the root which is getting google brigaded like many other critical comments in this thread. Also, this is Zuck level of malevolent naïveté, it’s not like they don’t have experience https://www.theverge.com/2024/2/21/24079371/google-ai-gemini-generative-inaccurate-historical https://www.theverge.com/2024/2/21/24079371/google-ai-gemini...
- programjames 2y agoI think the difference between our takes is our understanding of how the "AI Overview" was trained. Google said, > AI Overviews work very differently than chatbots and other LLM products that people may have tried out and later, > Let’s take a look at an example: “How many rocks should I eat?” Prior to these screenshots going viral, practically no one asked Google that question. It seems it's being trained based on search queries. This is important to protect against confabulation, but obviously AI will fail on out-of-distribution data. Such as search queries that practically no one asked about before. Now, they could ask an LLM if the AI Overview seems reasonable, but the entire point of their alternate training scheme is to guarantee the output can be backed with sources. Why introduce a potential failure mechanism? You say, "they gaslight us with words like 'faithful' and 'nobody asked the question,'" but given this is how their model works, it seems like the best, non-gaslighty defense. Plus, it's pretty unreasonable to expect Google to fix every potential bug before production. As they said, > there’s nothing quite like having millions of people using the feature with many novel searches. We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results. Having "a content policy violation on less than one in every 7 million unique queries" seems pretty solid to me. I'm sure it'd be much higher if they were using an LLM like Claude. EDIT: My argument is pretty much, it looks like they designed this to be helpful and accurate, so the errors seem like honest mistakes from imperfect execution rather than deception or politicking. Their post seems pretty open about acknowledging what went wrong and that they want to do better.
- qq66 2y agoThey don't seem to understand the problem. If they link you to a website that tells you to eat a rock, whatever. If they put their brand on that with an AI Overview, that's a problem.
- gmerc 2y agoThey understand just fine. It’s just they see the potential of running the equivalent of the world oracle where every monkey goes and receives the instructions from Gemini, the most powerful of AI deities as a native ad surface. They chose this. “Let them eat rocks” was a choice. After all they promised to do better after the racially diverse SS division dreamed up by Gemini a few months ago.
- season2episode3 2y agoMeanwhile, other LLM providers don’t need to worry as much about their brand image.
- beshrkayali 2y ago> We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results. Is this the first time they release something? Of course people are going to try to break it.
- gmerc 2y agoYea, they are new to this. https://www.theverge.com/2024/2/21/24079371/google-ai-gemini-generative-inaccurate-historical https://www.theverge.com/2024/2/21/24079371/google-ai-gemini...
- Workaccount2 2y agoMore interesting than the launch of AI overviews has been the universally cold and hostile response from the ad-supported media. There is an air of panicked existential dread in their reporting of it. Yeah the AI is new and rough. That will be fixed. What won't be fixed is people's desire to read AI overviews instead of click bait headlines that drag you through ad ridden filler before the one worthwhile point. The crack of Overview's faux pas is a twig breaking compared to the earth shaking crack of the words-for-adviews industry.
- xnx 2y agoThis is under-appreciated. AI overviews are an extinction level event for low-quality information-regurgitators. I'd love to read some analysis on this, since it seems like AI overviews will significantly erode Google's own AdSense revenue.
- Workaccount2 2y agoGoogle will almost certainly wield it's AI as a salesperson for advertisers. The ads will follow the users.
- gmerc 2y agoNo they are an extinction level event for all ad financed content. Google will just switch to “pay us to get listed” over time.
- prepend 2y ago> AI overviews are an extinction level event for low-quality information-regurgitators. I think this is good. They don’t serve a positive role in the information food chain. Freeing up their space should be helpful for people using information.
- mehulashah 2y agoThe shift from extracting content from the web and showing it versus generating content is an inflection point in search. It’s hard to get right, and I don’t agree that people like it better than regular search. It seems so, but the reality is more nuanced since positive engagement vs. frustration are hard to distinguish at a distance.
- xwolfi 2y agoIt's impossible to get it right: it's not search anymore, it's editorialising. And for computers to do it while adding value will take time, and probably more than applied statistics. Like maybe they should experience it themselves, go against the flow sometimes, have an actual, political even, opinion. A desire, an agenda. I see people asking the stuff for opinions, like "what should I do when X": you think it's search? You think it's "hard"? It's just insane and there will be a huge hangover once the limits have been properly understood. Puts on Nvidia. Google is whining that their billions of dollars invested in a thinking machine, an artificial "intelligence" in their own words, is unable to guess when it s been made a fool of, and gets a beautiful F on the Turing test, sigh: but at least it's more interesting than the "metaverse" virtual real estate grift I guess that their main advertising competitor attempted.
- gmerc 2y agoI’m waiting for the first section 230 case where they blame AI for the summary and argue it’s user generated content.
- xwolfi 2y agoWell I think in that case that'd be like your dog biting a naughty child in a park. It's not exactly entirely your fault, it's not the dog's fault, the kid is not entirely always innocent either. Put down the dog, fine the owner, scold the child and move on.
- yangcheng 2y agoAside from the content itself, the "Listen to Article" button uses a robotic, outdated TTS voice. Shouldn't a company like Google use their latest technologies in public-facing content, particularly when discussing AI progress? I'm genuinely curious about the decision-making process behind this choice.
- MarcScott 2y agoI'm pretty sure Brave has been doing this for awhile now, and it's certainly available outside of the US. It often saves me a click, which I appreciate.
- firloop 2y agoIt's amusing to me that they spend so much energy explaining that there were a lot of faked screenshots... but in this very same post they admit the tool told people to put glue on pizza and eat rocks!
- skybrian 2y agoWhat do you mean? Both can be true.
- firloop 2y agoOf course, but I would read it differently if the faked screenshots were wildly more absurd what was actually being generated (which in this case, they weren't!)
- gundmc 2y agoThe fake posts involved things like suggestions to jump off the Golden Gate Bridge and that cockroaches often live in human penises. Those were quite a bit more absurd (and dangerous) than the actual falsities.
- Hamuko 2y ago>Some of these faked results have been obvious and silly. Others have implied that we returned dangerous results for topics like leaving dogs in cars, smoking while pregnant, and depression. Meanwhile "add glue to your cheese" was a real result, which I guess was not Google "returning a dangerous result" because they specified that the glue should be non-toxic. Not really sure how obvious smoking while pregnant should be after that.
- piaste 2y agoThe quoted paragraph describes two distinct and _separate_ group: the "obvious and silly" ones, and the "others [which] implied that we returned dangerous results". Smoking while pregnant is listed among the latter.
- 2y ago
- Oras 2y agoThe article made me question Google search even more. « we tested the feature extensively before launch. This included robust red-teaming efforts, evaluations with samples of typical user queries and tests on a proportion of search traffic to see how it performed » Then she explains that: «We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results.» So, the red-teaming didn't think of prompt injections or how to identify fake results? If that's the case, wouldn't that mean all results from Google searches are questionable, considering how much crap coming out of SEO-farming websites?
- prepend 2y agoRegarding SEO stuff, I cynically think Google knows about it, but doesn’t do anything because it would reduce revenue. A friend told me that google estimates about 30% of its clicks are fraud, accident, waste, etc. But they have no incentive to fix because they make money off every wrong ad shown and every accidental or fraudulent click. So SEO-farming sites make Google money. If I have to click on different ads and multiple garbage sites, that’s more lucrative than just seeing a good result and skipping all the ads and blargh. Remember when “I feel lucky” was helpful? I can’t wait for some lawsuit discovery to produce the emails where google sees red team results for bad ads and decides to keep them because of revenue. This stuff doesn’t make google money, so they’re happy to cut it.
- greg_V 2y agoRe: the 30% fraud figure. An interesting book just came out last month that speculates on the amount of fraud / wasted clicks / impressions in the ecosystem https://dl.bookfunnel.com/6kwmi0qlsq https://dl.bookfunnel.com/6kwmi0qlsq TL;DR it's pretty damning, but since ultimately advertising is shown to drive revenue, the whole problem is priced in so to speak.