3 ms·

I’ve been using this for a while on my mastodon server and after a few tweaks to make sure it wasn’t blocking legit traffic it’s been really working great. Betw

I’ve been using this for a while on my mastodon server and after a few tweaks to make sure it wasn’t blocking legit traffic it’s been really working great. Between Microsoft and Meta, they were hitting my services more than any other traffic combined which says a lot of you know how noisy mastodon can be. Server load went down dramatically. It also completely put a stop to perplexity as far as I can tell. And the robots file meant nothing, they’d still request it hundreds of thousands of times instead of caching it. Every request they’d hit it first then hit their intended url.

TechDebtDevin 1y ago
This does nothing dude. Literally nothing. OpenAI or whoever are just going to hire people like me who dont get caught. Stop ruining the experience of users and allowing cf to fill the internet with more bloated javascript challenge pages and privacy invading fingerprinting. Stop making cf the police of the internet. We're literally handing the internet to this company on a silver platter to do MITM attacks on our privacy and god knows what else. Fucking wild.
- fluidcruft 1y ago
  They literally said it significantly reduced their server resource usage. Are you suggesting they are lying?
- drowsspa 1y ago
  Why do you think you have the moral high ground here?
- dawnerd 1y ago
  Well the alternative is to not have an instance at all so… what do you suggest? I’m not paying for the other services, it’s already expensive enough to run the site. The goal isn’t to stop 100% of scrapers, it was to reduce server load to a level that wasn’t killing the site.
- jowea 1y ago
  You want them to pay the server costs to serve content to AI scrappers for free? The alternative is Anubis, which is maybe equally annoying to users in a different way.
danielspace23 1y ago
Have you considered Anubis? I know it's harder to install, but personally, I think the point of Mastodon is trying to avoid centralization where possible, and CloudFlare is one of the corporations that are keeping the internet centralized.
- dawnerd 1y ago
  Haven’t heard of it, will look into it. I agree, I’d rather not have cloudflare but for what they provide for free it’s a tough offer to pass up

TechDebtDevin 1y ago

This does nothing dude. Literally nothing. OpenAI or whoever are just going to hire people like me who dont get caught. Stop ruining the experience of users and allowing cf to fill the internet with more bloated javascript challenge pages and privacy invading fingerprinting. Stop making cf the police of the internet. We're literally handing the internet to this company on a silver platter to do MITM attacks on our privacy and god knows what else. Fucking wild.

fluidcruft 1y ago
They literally said it significantly reduced their server resource usage. Are you suggesting they are lying?
drowsspa 1y ago
Why do you think you have the moral high ground here?
dawnerd 1y ago
Well the alternative is to not have an instance at all so… what do you suggest? I’m not paying for the other services, it’s already expensive enough to run the site. The goal isn’t to stop 100% of scrapers, it was to reduce server load to a level that wasn’t killing the site.
jowea 1y ago
You want them to pay the server costs to serve content to AI scrappers for free? The alternative is Anubis, which is maybe equally annoying to users in a different way.

danielspace23 1y ago

Have you considered Anubis? I know it's harder to install, but personally, I think the point of Mastodon is trying to avoid centralization where possible, and CloudFlare is one of the corporations that are keeping the internet centralized.

dawnerd 1y ago
Haven’t heard of it, will look into it. I agree, I’d rather not have cloudflare but for what they provide for free it’s a tough offer to pass up