5 ms·

I'm on AWS and use their WAF service to do some rudimentary bot blocking, but CDNs (their Cloudflare equivalent, Cloudfront) have been too expensive in the past

by dougb5 2y ago

I'm on AWS and use their WAF service to do some rudimentary bot blocking, but CDNs (their Cloudflare equivalent, Cloudfront) have been too expensive in the past and the bot control mechanisms they offer have too many false positives. Perhaps I should check it out again. Part of the problem is the economics of it -- I've chosen to self-fund a high traffic site without ads, and that's on me. But it was possible to do this just a few years ago.

gruez 2y ago
>and the bot control mechanisms they offer have too many false positives. Perhaps I should check it out again. Cloudflare no longer does CAPTCHAs so even if users get flagged as bots, the user experience isn't terrible. You just have to click on a box and you're on your way. It adds maybe 3s of delay, far better than anti-bot solutions that require you to solve an captcha, or imperva's (?) challenge that requires you to hold a button for 5-10s seconds.
- lcampbell 2y ago
  If you're given a button to click, your browser has successfully passed the environment integrity checks and you have not been flagged as a bot. You'll be flagged as a bot if your browser configuration has something "weird" (e.g. webrtc is disabled to reduce your attack surface) and you will be completely unable to access any site behind cloudflare with the anti-bot options turned on. You'll get an infinite redirect loop, not a button to click.
  - immibis 2y ago
    Note that Google's version of this was determined to be checking whether you had a 9-day-old tracking cookie. The researcher who discovered this was able to generate 60,000 "I am not a bot" cookies per day, and use them up about 15 times each in a bot before it started getting captchas. That was in 2016 though.
    - rixed 2y ago
      Couldn't the content of that cookie be used to validate its actual age? Like, just signing the date of generation?
      - gruez 2y ago
        ...or generate a bunch and wait 9 days?
      - immibis 2y ago
        That's probably what it was. So they accessed some page over and over, pretending to not have the cookie yet, got a bunch of cookies, and 9 days later, used them to bypass captchas.
- dougb5 2y ago
  That is helpful, thank you! I will check it out.
linuxftw 2y ago
Cloudflare is free. Give it a try.
- leviathant 2y ago
  I'm not sure why this is down voted. The forum I run was hitting monthly bandwidth limits within days because of bots. Changes to htaccess and checking environment data in the PHP code was a cat and mouse game. The free Cloudflare tier solved my problem.

by dougb5 2y ago

gruez 2y ago

>and the bot control mechanisms they offer have too many false positives. Perhaps I should check it out again. Cloudflare no longer does CAPTCHAs so even if users get flagged as bots, the user experience isn't terrible. You just have to click on a box and you're on your way. It adds maybe 3s of delay, far better than anti-bot solutions that require you to solve an captcha, or imperva's (?) challenge that requires you to hold a button for 5-10s seconds.

lcampbell 2y ago
If you're given a button to click, your browser has successfully passed the environment integrity checks and you have not been flagged as a bot. You'll be flagged as a bot if your browser configuration has something "weird" (e.g. webrtc is disabled to reduce your attack surface) and you will be completely unable to access any site behind cloudflare with the anti-bot options turned on. You'll get an infinite redirect loop, not a button to click.
- immibis 2y ago
  Note that Google's version of this was determined to be checking whether you had a 9-day-old tracking cookie. The researcher who discovered this was able to generate 60,000 "I am not a bot" cookies per day, and use them up about 15 times each in a bot before it started getting captchas. That was in 2016 though.
  - rixed 2y ago
    Couldn't the content of that cookie be used to validate its actual age? Like, just signing the date of generation?
    - gruez 2y ago
      ...or generate a bunch and wait 9 days?
    - immibis 2y ago
      That's probably what it was. So they accessed some page over and over, pretending to not have the cookie yet, got a bunch of cookies, and 9 days later, used them to bypass captchas.
dougb5 2y ago
That is helpful, thank you! I will check it out.

linuxftw 2y ago

Cloudflare is free. Give it a try.

leviathant 2y ago
I'm not sure why this is down voted. The forum I run was hitting monthly bandwidth limits within days because of bots. Changes to htaccess and checking environment data in the PHP code was a cat and mouse game. The free Cloudflare tier solved my problem.