5 ms·

The headline is somewhat misleading: sites using Cloudflare now have an opt-in option to quickly block all AI bots, but it won't be turned on by default for sit

by btown 1y ago

The headline is somewhat misleading: sites using Cloudflare now have an opt-in option to quickly block all AI bots, but it won't be turned on by default for sites using Cloudflare. The idea that Cloudflare could do the latter at the sole discretion of its leadership, though, is indicative of the level of power Cloudflare holds.

bitpush 1y ago
It is now an adversarial relationship between aibots and website, and cloudflare is merely reacting to it. Would you say the same for ddos protection? Isn't that the same as well?
- TechDebtDevin 1y ago
  They arent doing anything. They are attempting to insert themselves into the middle of a marketplace (that doesnt exist and never will) where scrapers pay for IP. They think theyre going to profit off the bots, not protect your site. Dont fall for their scam.
  - bitpush 1y ago
    What do you mean they are trying to insert themselves? If I have a website that I host with cloudflare, I (as the rightful website owner) has inserted Cloudflare in between. It isnt CF going around saying, that's a nice website you have there. I'm gonna put myself in between.
    - deleted 1y ago
      [deleted]
GrayShade 1y ago
> sites using Cloudflare now have an opt-in option to quickly block all AI bots, but it won't be turned on by default for sites using Cloudflare Do you have a source for that? https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/ https://blog.cloudflare.com/content-independence-day-no-ai-c... does say "changing the default".
- mattcollins 1y ago
  "This feature is available to all customers, meaning anyone can enable this today from the Cloudflare dashboard." https://blog.cloudflare.com/control-content-use-for-ai-training/#putting-up-a-guardrail-with-cloudflares-managed-robots-txt https://blog.cloudflare.com/control-content-use-for-ai-train...
TechDebtDevin 1y ago
They cant do anything other than bog down the internet. I havent found a single cf provided challenge I havent been able to get past in < half a day. This is simply juat the first step in them implementing a marketplace and trying to get into LLM SEO. They dont care about your site or protecting it. They are gearing up to start making a cut in the Middle between scrapers and publishers. Why wouldnt I go DIRECTLY to the publisher and make a deal. So dumb I hate cf so much. The only thing cloudflare knows how to do is MITM attacks.
- Marsymars 1y ago
  So what would you suggest as an alternative if I have a site where I don’t want the content used for LLM training?
  - fkyoureadthedoc 1y ago
    Auth? Because whatever Cloudflare is doing isn't going to stop anyone serious about scraping data.
    - Marsymars 1y ago
      Let’s say I’m talking about content that I don’t want behind an auth wall. Is your position simply that all such sites should abandon any efforts to not have the content used for LLM training?
      - mattl 1y ago
        If you find a solution that’s not auth please let me know.
      - Wilder7977 1y ago
        Something like https://github.com/TecharoHQ/anubis https://github.com/TecharoHQ/anubis? It's not that different from CF, but you control it fully.
      - fkyoureadthedoc 1y ago
DeusExMachina 1y ago
I would expect these features to be opt-in. Even though I agree with it, I would be pretty upset if they just turned it on automatically on my website.

by btown 1y ago

bitpush 1y ago

It is now an adversarial relationship between aibots and website, and cloudflare is merely reacting to it. Would you say the same for ddos protection? Isn't that the same as well?

TechDebtDevin 1y ago
They arent doing anything. They are attempting to insert themselves into the middle of a marketplace (that doesnt exist and never will) where scrapers pay for IP. They think theyre going to profit off the bots, not protect your site. Dont fall for their scam.
- bitpush 1y ago
  What do you mean they are trying to insert themselves? If I have a website that I host with cloudflare, I (as the rightful website owner) has inserted Cloudflare in between. It isnt CF going around saying, that's a nice website you have there. I'm gonna put myself in between.
  - deleted 1y ago
    [deleted]

GrayShade 1y ago

> sites using Cloudflare now have an opt-in option to quickly block all AI bots, but it won't be turned on by default for sites using Cloudflare Do you have a source for that? https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/ https://blog.cloudflare.com/content-independence-day-no-ai-c... does say "changing the default".

mattcollins 1y ago
"This feature is available to all customers, meaning anyone can enable this today from the Cloudflare dashboard." https://blog.cloudflare.com/control-content-use-for-ai-training/#putting-up-a-guardrail-with-cloudflares-managed-robots-txt https://blog.cloudflare.com/control-content-use-for-ai-train...

TechDebtDevin 1y ago

They cant do anything other than bog down the internet. I havent found a single cf provided challenge I havent been able to get past in < half a day. This is simply juat the first step in them implementing a marketplace and trying to get into LLM SEO. They dont care about your site or protecting it. They are gearing up to start making a cut in the Middle between scrapers and publishers. Why wouldnt I go DIRECTLY to the publisher and make a deal. So dumb I hate cf so much. The only thing cloudflare knows how to do is MITM attacks.

Marsymars 1y ago
So what would you suggest as an alternative if I have a site where I don’t want the content used for LLM training?
- fkyoureadthedoc 1y ago
  Auth? Because whatever Cloudflare is doing isn't going to stop anyone serious about scraping data.
  - Marsymars 1y ago
    Let’s say I’m talking about content that I don’t want behind an auth wall. Is your position simply that all such sites should abandon any efforts to not have the content used for LLM training?
    - mattl 1y ago
      If you find a solution that’s not auth please let me know.
    - Wilder7977 1y ago
      Something like https://github.com/TecharoHQ/anubis https://github.com/TecharoHQ/anubis? It's not that different from CF, but you control it fully.
    - fkyoureadthedoc 1y ago

DeusExMachina 1y ago

I would expect these features to be opt-in. Even though I agree with it, I would be pretty upset if they just turned it on automatically on my website.