4 ms·

Cloudflare offers bot mitigation for free, and pretty generous WAF rules that makes mitigations like this seem a little overblown to me

n1xis10t 10mo ago
You can’t deny that it’s fun though. Personally I generally feel like more people should be coming up with creative (if not entirely necessary) solutions to problems.
conception 10mo ago
For “free”.
- n1xis10t 10mo ago
  Did you put “free” in quotes because you need to have paid for stuff from cloudflare to use the “free” thing? If so, I suppose it’s like those magazines that say ”free cd”.
  - Terr_ 10mo ago
    I thought they were referring to the indirect costs of supporting monopolistic stuff that enshittifies later. https://www.youtube.com/watch?v=U8vi6Hbp8Vc https://www.youtube.com/watch?v=U8vi6Hbp8Vc
  - JohnMakin 10mo ago
    You don't though.
    - n1xis10t 10mo ago
      Good to know thanks
  - efilife 10mo ago
    Well, you literally MITM yourself so I think it's a big price
ATechGuy 10mo ago
It is really free? Genuinely asking.
- gilrain 10mo ago
  Yes. They upsell more complete solutions, but the free tier is pretty generous.
nospice 10mo ago
I'm on the free tier, but I also watch my logs. The vast majority of the traffic I'm getting are scrapers and vulnerability scanners, a lot of them coming through residential proxies and other "laundered" egress points. I honestly don't think that Cloudflare is on top of the problem at all. They claim to be blocking abuse, but in my experience, most of the badness gets through.
- cakealert 10mo ago
  when you combine a residential proxy with a tool like curl-impersonate (there are libraries in Go for this type of fingerprint spoofing now) they dont even show up as scrapers anymore, just users. especially when they adjust timings to mimic humans. clouflare only blocks the most dumb of bots, there are still a lot of them. this is why cloudflare will issue javascript challenges to you even when you are using google chrome with a VPN, they are desperate to appear to be doing something. and every VPN is used to crawl as well. a slightly more sophisticated bot passes the cloudflare javascript challenge as well, there really is nothing they can do to win here. i know some teams that got annoyed with residential proxies (they are usually sold as socks5 but can be buggy and low bandwidth) so they invested into defeating the cloudflare javascript challenge and now crawl using 1000's of VPN endpoints at over 100 Gbit/s.
  - oidar 10mo ago
    Is "residential proxy" another name for an hacked/owned computer that the bots have access to? Or are there legitimate services that sell access to residential IPs?
    - nospice 10mo ago
      People legitimately sell egress. It's "free" money. But of course, if you have a botnet, you can sell that through the same channels, no one is looking too closely.

n1xis10t 10mo ago

You can’t deny that it’s fun though. Personally I generally feel like more people should be coming up with creative (if not entirely necessary) solutions to problems.

conception 10mo ago

For “free”.

n1xis10t 10mo ago
Did you put “free” in quotes because you need to have paid for stuff from cloudflare to use the “free” thing? If so, I suppose it’s like those magazines that say ”free cd”.
- Terr_ 10mo ago
  I thought they were referring to the indirect costs of supporting monopolistic stuff that enshittifies later. https://www.youtube.com/watch?v=U8vi6Hbp8Vc https://www.youtube.com/watch?v=U8vi6Hbp8Vc
- JohnMakin 10mo ago
  You don't though.
  - n1xis10t 10mo ago
    Good to know thanks
- efilife 10mo ago
  Well, you literally MITM yourself so I think it's a big price

ATechGuy 10mo ago

It is really free? Genuinely asking.

gilrain 10mo ago
Yes. They upsell more complete solutions, but the free tier is pretty generous.

nospice 10mo ago

I'm on the free tier, but I also watch my logs. The vast majority of the traffic I'm getting are scrapers and vulnerability scanners, a lot of them coming through residential proxies and other "laundered" egress points. I honestly don't think that Cloudflare is on top of the problem at all. They claim to be blocking abuse, but in my experience, most of the badness gets through.

cakealert 10mo ago
when you combine a residential proxy with a tool like curl-impersonate (there are libraries in Go for this type of fingerprint spoofing now) they dont even show up as scrapers anymore, just users. especially when they adjust timings to mimic humans. clouflare only blocks the most dumb of bots, there are still a lot of them. this is why cloudflare will issue javascript challenges to you even when you are using google chrome with a VPN, they are desperate to appear to be doing something. and every VPN is used to crawl as well. a slightly more sophisticated bot passes the cloudflare javascript challenge as well, there really is nothing they can do to win here. i know some teams that got annoyed with residential proxies (they are usually sold as socks5 but can be buggy and low bandwidth) so they invested into defeating the cloudflare javascript challenge and now crawl using 1000's of VPN endpoints at over 100 Gbit/s.
- oidar 10mo ago
  Is "residential proxy" another name for an hacked/owned computer that the bots have access to? Or are there legitimate services that sell access to residential IPs?
  - nospice 10mo ago
    People legitimately sell egress. It's "free" money. But of course, if you have a botnet, you can sell that through the same channels, no one is looking too closely.