3 ms·

Apparently, the regular search crawler does it, but the ai thingie doesn't.

• supriyo-biswas 2y ago
  Huh? You can add Google-Extended[1] to opt out from Generative AI summaries. [1] https://blog.google/technology/ai/an-update-on-web-publisher-controls/ https://blog.google/technology/ai/an-update-on-web-publisher...
  • micromacrofoot 2y ago
    Google will still scrape it for training data either way, this only impacts search results.
    • supriyo-biswas 2y ago
      > Today we’re announcing Google-Extended, a new control that web publishers can use to manage whether their sites help *improve Bard and Vertex AI generative APIs*, including future generations of models that power those products.
      • micromacrofoot 2y ago
        https://www.theverge.com/news/630079/openai-google-copyright-fair-use-exception https://www.theverge.com/news/630079/openai-google-copyright... they're literally asking to break laws to train AI for national security. A sentence in a press release from 2 years ago is worthless... look at what they're actually doing
• lucgagan 2y ago
  Can confirm. My website is flooded with AI bots despite attempts to block crawlers to certain parts of it.