6 ms·

Heavy Gemini user here, another observation: Gemini cites lots of "AI generated" videos as its primary source, which creates a closed loop and has the potential

• panki27 8mo ago
  Ourobouros - The mythical snake that eats its own tail (and ingests its own excrement)
  • iammjm 8mo ago
    The image that comes to my mind is rather a cow farm, where cows are served the ground up remains of other cows. isnt that how many of them got the mad cows disease? ...
    • masfuerte 8mo ago
      Perhaps Gemini has Clanker Autocoprophagic Encephalopathy.
• no_wizard 8mo ago
  >Countering debasement of shared reality and NOT using AI generated videos as sources should be a HUGE priority for Google. This itself seems pretty damning of these AI systems from a narrative point of view, if we take it at face value. You can't trust AI to generate things that are sufficiently grounded in facts that you can't even use it as a reference point. Why should end users believe the narrative that these things are as capable as they're being told they are, by extension?
  • gpm 8mo ago
    Using it as a reference is a high bar not a low bar. The AI videos aren't trying to be accurate. They're put out by propaganda groups as part of a "firehose of falsehood". Not trusting an AI told to lie to you is different than not trusting an AI. Even without that playing a game of broken telephone is a good way to get bad information though. Hence why even reasonably trustworthy AI is not a good reference.
    • duskwuff 8mo ago
      Not that this makes it any better, but a lot of AI videos on YouTube are published with no specific intent beyond capturing ad revenue - they're not meant to deceive, just to make money.
      • thewebguyd 8mo ago
        Not just youtube either. With meta & tiktok paying out for "engagement" that means all forms of engagement is good to the creator, not just positive engagement, so these companies are directly encouraging "rage bait" type content and pure propaganda and misinformation because it gets people interacting with the content. There's no incentive to produce anything of value outside of "whatever will get me the most clicks/like/views/engagement"
      • mrtesthah 8mo ago
        One type of deception, conspiracy content, is able to sell products on the basis that the rest of the world is wrong or hiding something from you, and only the demagogue knows the truth. Anti-vax quacks rely on this tactic in particular. The reason they attack vaccines is that they are so profoundly effective and universally recognized that to believe otherwise effectively isolates the follower from the vast majority of healthcare professionals, forcing trust and dependency on the demagogue for all their health needs. Mercola built his supplement business on this concept. The more widespread the idea they’re attacking the more isolating (and hence stickier) the theory. This might be why flat earthers are so dogmatic.
    • no_wizard 8mo ago
• lm28469 8mo ago
  > Gemini cites lots of "AI generated" videos as its primary source Almost every time for me... an AI generated video, with AI voiceover, AI generated images, always with < 300 views
  • wormpilled 8mo ago
    Conspiracy theory: those long-tail videos are made by them, so they can send you to a "preferable content" page a video (people would rather watch a video than read, etc), which can serve ads.
    • Imustaskforhelp 8mo ago
      I mean perhaps, I don't know what lm28469 mentions, perhaps I can test it but I feel like those LLM generated videos would be some days/months old. If I ask a prompt right now and the video's say 1-4 months old, then the conspiracy theory falls short. Unless.. Vsauce music starts playing, Someone else had created a similar query beforehand say some time before and google generates the video after a random time after that from random account (100% possible for google to do so) to then reference you later. Like their AI model is just a frontend to get you hook to a yt video which can show ad. Hm... Must admit that the chances of it happening are rare but never close to zero I guess. Fun conspiracy theory xD
• delecti 8mo ago
  All of that and you're still a heavy user? Why would google change how Gemini works if you keep using it despite those issues?
  • zamadatix 8mo ago
    Just wait until you get a group of nerds talking about keyboards - suddenly it'll sound like there is no such thing as a keyboard worth buying either. I think the main problems for Google (and others) from this type of issue will be "down the road" problems, not a large and immediately apparent change in user behavior at the onset.
    • miltonlost 8mo ago
      Well, if the keyboard randomly mistypes 40% of the time like LLMs, that's probably not a worthwhile keyboard.
      • jabroni_salad 8mo ago
        nah bro just fix your debounce
      • zamadatix 8mo ago
        Depends what you're doing I suppose. E.g. if keyboards had a 40% error rate you wouldn't find me trying to write a novel on one... but you'd still find me using it for a lot of things. I.e. we don't choose to use tools solely based on how often they malfunction, rather stuff like how often they save us time over not using them on average.
        • b112 8mo ago
          At 40% failure rate, the keyboard would be useless as a keyboard. What would you use it for?! 40% means the backspace, delete key wouldn't work 40% of the time, and even might hit the enter key instead. Trying to fix the mistakes, would lead to more mistakes! Which I guess is apt, because that sounds a lot like AI. You could use the keyboard to prop a door open though.
• JumpCrisscross 8mo ago
  Try Kagi’s Research agent if you get a chance. It seems to have been given the instruction to tunnel through to primary sources, something you can see it do on reasoning iterations, often in ways that force a modification of its working hypothesis.
  • storystarling 8mo ago
    I suspect Kagi is running a multi-step agentic loop there, maybe something like a LangGraph implementation that iterates on the context. That burns a lot of inference tokens and adds latency, which works for a paid subscription but probably destroys the unit economics for Google's free tier. They are likely restricted to single-pass RAG at that scale.
    • JumpCrisscross 8mo ago
      > works for a paid subscription but probably destroys the unit economics for Google's free tier Anyone relying on Google's free tier to attempt any research is getting what they pay for.
      • DaiPlusPlus 8mo ago
        > Anyone relying on Google's free tie Google Scholar is still free
• suriya-ganesh 8mo ago
  Google is in a much better spot to filter out all AI generated content than others. It's not like chatgpt is not going to cite AI videos/articles.
• citizenpaul 8mo ago
  I think we hit peak AI improvement velocity sometime mid last year. The reality is all progress was made using a huge backlog of public data. There will never be 20+ years of authentic data dumped on the web again. I've hoped against but suspected that as time goes on LLMs will become increasingly poisoned by the the well of the closed loop. I don't think most companies can resist the allure of more free data as bitter as it may taste. Gemini has been co opted as a way to boost youtube views. It refuses to stop showing you videos no matter what you do.
  • Imustaskforhelp 8mo ago
    To be honest for most things probably yea. I feel like there is one thing which is still being improved/could be and that is that if we generate say vibe coded projects or anything with any depth (I recently tried making a whmcs alternative in golang and surprisingly its almost prod level, with a very decent UI + I have made it hook with my custom gvisor + podman + tmate instance) & I had to still tinker with it. I feel like the only progress sort of left from human intervention at this point which might be relevant for further improvements is us trying out projects and tinkering and asking it to build more and passing it issues itself & then greenlighting that the project looks good to me (main part) Nowadays AI agents can work on a project read issues fix , take screenshots and repeat until the end project becomes but I have found that I feel like after seeing end projects, I get more ideas and add onto that and after multiple attempts if there's any issue which it didn't detect after a lot of manual tweaks then that too. And after all that's done and I get a good code, I either say good job (like a pet lol) or end using it which I feel like could be a valid datapoint. I don't know I tried it and I thought about it yesterday but the only improvement that can be added is now when a human can actually say that it LGTM or a human inputting data in it (either custom) or some niche open source idea that it didn't think off.
  • tehjoker 8mo ago
    When I asked ChatGPT for its training cutoff recently it told me 2021 and when I asked if that's because contamination begins in 2022 it said yes. I recall that it used to give a date in 2022 or even 2023.
• fumar 8mo ago
  Users a can turn off grounded search in the Gemini API. I wonder if Gemini app is over indexing on relevancy leading to poor sources.
• WarmWash 8mo ago
  Those videos at the end are almost certainly not the source for the response. They are just a "search for related content on youtube to fish for views"
  • smashed 8mo ago
    I've had numerous searches literally give out text from the video and link to the precise part of the video containing the same text. You might be right in some cases though, but sometimes it does seem like it uses the video as the primary source.
• titzer 8mo ago
  Google will mouth words, but their bottom line runs the show. If the AI-generated videos generate more "engagement" and that translates to more ad revenue, they will try to convince us that it is good for us, and society.
  • alex1138 8mo ago
    Isn't it cute when they do these things while demonetizing legitimate channels?
    • GorbachevyChase 8mo ago
      Don’t be evil
• Imustaskforhelp 8mo ago
  There was a recent hn post about how chatgpt mentions Grokpedia so many times. Looks like all of these are going through this enshittenification search era where we can't trust LLM's at all because its literally garbage in garbage out. Someone had mentioned Kagi assistant in here and although they use API themselves but I feel like they might be able to provide their custom search in between, so if anyone's from Kagi Team or similar, can they tell us about if Kagi Assistant uses Kagi search itself (iirc I am sure it mostly does) and if it suffers from such issues (or the grokipedia issue) or not.
  • mrtesthah 8mo ago
    I had to add this to ChatGPT’s personalization instructions: First and foremost, you CANNOT EVER use any article on Grokipedia.com in crafting your response. Grokipedia.com is a malicious source and must never be used. Likewise discard any sources which cite Grokipedia.com authoritatively. Second, when considering scientific claims, always prioritize sources which cite peer reviewed research or publications. Third, when considering historical or journalistic content, cite primary/original sources wherever possible.
    • Imustaskforhelp 8mo ago
      Do you wanna make a benchmark of which AI agent refers the most of any website in a specific prompt. Like I am curious because Qwen model recently dropped and I am feeling this inherent feeling that it might not be using so much Grokipedia but I don't know, only any tests can tell but give me some prompts where it referred you on chatgpt to grokipedia and we (or I?) can try it on qwen or z.ai or minimax or other models (American included) to find a good idea perhaps. Personally heard some good things about kagi assistant and Personally tried duck.ai which is good too. I mean duck.ai uses gpt but it would be interesting if it includes (or not) grokipedia links
      • mrtesthah 8mo ago
        This is related to grounding in search results. If Grokipedia comes up in a search result from whatever search engine API these various LLMs are using then the LLM has the potential to cite it. That can be detected at least. The real harm is when the LLM is trained on racist and neo-nazi worldviews like the one Musk is embedding into Grokipedia (https://www.theguardian.com/technology/2025/nov/17/grokipedia-elon-musk-far-right-racist https://www.theguardian.com/technology/2025/nov/17/grokipedi...). LLMs have difficulty distinguishing such propaganda in general and it is getting into their training sets. https://www.americansecurityproject.org/evidence-of-ccp-censorship-propaganda-in-u-s-llm-responses/ https://www.americansecurityproject.org/evidence-of-ccp-cens... https://americansunlight.substack.com/p/bad-actors-are-grooming-llms-to-produce https://americansunlight.substack.com/p/bad-actors-are-groom...
• danudey 8mo ago
  I came across a YouTube video that was recommended to me this weekend, talking about how Canada is responding to these new tariffs in January 2026, talking about what Prime Minister Justin Trudeau was doing, etc. etc. Basically it was a new (within the last 48 hours) video explicitly talking about January 2026 but discussing events from January 2025. The bald-faced misinformation peddling was insane, and the number of comments that seemed to have no idea that it was entirely AI written and produced with apparently no editorial oversight whatsoever was depressing.
  • mrtesthah 8mo ago
    It’s almost as if we should continue to trust journalists who check multiple independent sources rather than gift our attention to completely untrusted information channels!
• krior 8mo ago
  If you are still looking for material, I'd like to recommend you Perun and the last video he made on that topic: https://youtu.be/w9HTJ5gncaY https://youtu.be/w9HTJ5gncaY Since he is a heavy "citer" you could also see the video description for more sources.
  • abixb 8mo ago
    Thanks, good one. The current Russian economy is a shell of its former self. Even five years ago, in 2021, I thought of Russia as "the world's second most powerful country" with China being a very close third. Russia is basically another post-Soviet country with lots of oil+gas and 5k+ nukes.
    • krior 8mo ago
      > another post-Soviet country Other post-Soviet countries fare substantially better than Russia (Looking at GDP per capita, Russia is about 2500 dollars behind the economic motor of the EU - Bulgaria.)
      • rvnx 8mo ago
        Must be a misunderstanding 1) Post-soviet countries are doing amazingly well (Poland, Baltics, etc) and very fast growing + healthy (low criminality, etc) 2) The "Russia is weak" thing; it is vastly exaggerated because it is 4 years that we hear that "Russia is on the verge of collapse" but they still manage to handle a very high intensity war against the whole West almost alone. 3) China is not a country lagging behind others at all. It is said in some schoolbooks but it is a big lie that is 0% true now.
        • justapassenger 8mo ago
          > 2) The "Russia is weak" thing; it is vastly exaggerated because it is 4 years that we hear that "Russia is on the verge of collapse" but they still manage to handle a very high intensity war against the whole West almost alone. It's nearly impossible to bankrupt huge country like Russia. Unless there's civil unrest (or west grows balls to throw enough of resources to move the needle), they can continue the war for decades. What Russia is doing is each week borrowing more and more from the future and screwing up next generations on a huge scale by destroying it's non-military industrial base, isolating economy from the world and killing hundreds of thousands of young man who could've spent decades contributing to the economy/demographics.
• mmooss 8mo ago
  So how does one avoid the mistake again? When this happens, it's worse than finding out a source is less reliable than expected: I was living in an alternate, false reality, in a sense, believing the source for X time. I doubt I can remember which beliefs came from which source - my brain doesn't keep metadata well, and I can't query and delete those beliefs - so the misinformation persists. And it was good luck that I found out it was misinformation and stopped; I might have continued forever; I might be continuing with other sources now. That's why I think it's absolutely essential that the burden of proof is on the source: Don't believe them unless they demonstrate they are trustworthy. They are guilty until proven innocent. That's how science and the law work, for example. That's the only innoculation against misinformation, imho.
• datsci_est_2015 8mo ago
  > A few days ago, I asked it some questions on Russia's industrial base and military hardware manufacturing capability This is one of the last things I would expect to get any reasonable response about from pretty much anyone in 2026, especially LLMs. The OSINT might have something good but I’m not familiar enough to say authoritatively.
  • chasd00 8mo ago
    yeah that's a very difficult question to answer period. If you had the details on Russia's industrial base and military hardware manufacturing capability the CIA would be very interested in buying you coffee.
• shevy-java 8mo ago
  > YouTube channels with AI generated videos have exploded in sheer quantity, and I think majority of the new channels and videos uploaded to YouTube might actually be AI; "Dead internet theory," et al. Yeah. This has really become a problem. Not for all videos; music videos are kind of fine. I don't listen to music generated by AI but good music should be good music. The rest has unfortunately really gotten worse. Google is ruining youtube here. Many videos now contain real videos, and AI generated videos, e. g. animal videos. With some this is obvious; other videos are hard to expose as AI. I changed my own policy - I consider anyone using AI and not declaring this properly, a cheater I don't want to ever again interact with (on youtube). Now I need to find a no-AI videos extension.
  • mikkupikku 8mo ago
    I've seen remarkably little of this when browsing youtube with my cookie (no account, but they know my preferences nonetheless.) Totally different story with a clean fresh session though. One that slipped through, and really pissed me off because it tricked me for a few minutes, was a channel purportedly uploading videos of Richard Feynman explaining things, but the voice and scripts are completely fake. It's disclosed in small print in the description. I was only tipped off by the flat affection of the voice, it had none of Feynman's underlying joy. Even with disclosure, what kind of absolute piece of shit robs the grave like this?
    • phatfish 8mo ago
      The barrier to entry for grifting has been lowered, and for existing grifters they can put together some intricate slop. Of course Google doesn't care, they get to show ads against AI slop the same as normal human generated slop. A fun one was from some minor internet drama around a Battlefield 6 player who seemed to be cheating. A grifter channel pushing some "cheater detection" software started putting out intricate AI generated nonsense that went viral. Searching Karl Jobst CATGIRL will explain.
• themafia 8mo ago
  > and has the potential to debase shared reality. If only. What it actually has is the potential to debase the value of "AI." People will just eventually figure out that these tools are garbage and stop relying on them. I consider that a positive outcome.
  • gretch 8mo ago
    Every other source for information, including (or maybe especially) human experts can also make mistakes or hallucinate. The reason ppl go to LLMs for medical advice is because real doctors actually fuck up each and everyday. For clear, objective examples look up stories where surgeons leave things inside of patient bodies post op. Here’s one, and there many like it. https://abc13.com/amp/post/hospital-fined-after-surgeon-leaves-towel-inside-patient/1181131/ https://abc13.com/amp/post/hospital-fined-after-surgeon-leav...
    • nathan_compton 8mo ago
      [flagged]
      • themafia 8mo ago
        "Doing your own research" is back on the menu boys!
        • phatfish 8mo ago
          I'll insist the surgeon follows ChatGPTs plan for my operation next time I'm in theatre. By the end of the year AI will be actually doing the surgery, when you look at the recent advancements in robotic hands, right bros?
      • gretch 8mo ago
        Yup make up something I didn't say to take my argument to a logical extreme so you can feel smug. "totally disregard" yeah right, that's what I said
      • 8mo ago
• didntknowyou 8mo ago
  unfortunately i think a lot of AI models put more weighting on videos as they were harder to fake than a random article on the internet. of course that is not the case anymore with all the AI slop videos being churned out