15 ms·

The problem presented by services like ReCaptcha and Cloudflare is a tough nut to crack. They're silently embedded in a huge portion of modern websites, and th

by Liquix 7y ago

The problem presented by services like ReCaptcha and Cloudflare is a tough nut to crack. They're silently embedded in a huge portion of modern websites, and the average user will never even know about them. But it seems to be way too easy for them to blanket-ban or serve an absurd amount of captchas to powerusers, linux gurus, privacy geeks, or anyone with the wrong combination of browser+addons. And the failures (as in this case) are often silent, cryptic, un-fixable from the user end, and can prevent us from accessing massive swaths of the internet. Any thoughts surrounding this conundrum? Solutions: 1. Everyone stops using ReCaptcha/Cloudflare. - Never going to happen. They dominate the market because they are useful, well-made services. 2. Launch a competing product that accomplishes the same thing. - Good luck competing with these giants. Also, how would your implementation differ to solve this issue? 3. Powerusers and tech nerds must conform to 'normal' browser configurations and disable privacy addons in order to enjoy the internet with 'normal' users. - Two steps backwards in every conceivable way. The giants gain more invisible power and powerusers suffer decreased productivity/privacy. Not going to happen.

sturgill 7y ago
I’m a fan of Recaptcha v3. There are many actions where you can ask for some additional input in a non-puzzle way. Simple example is sending a confirmation email before signup when the score is below a certain threshold. Because I freaking hate those captcha puzzles...
- rstupek 7y ago
  Except v3 has to be on every page of the site and that gives Google a full view of all traffic on the site
  - WillPostForFood 7y ago
    So does Google Analytics, so does AdSense. ReCaptcha V3 is probably the least used of the three.
    - catalogia 7y ago
      Blocking GA and AdSense doesn't render websites unusable.
    - ddalex 7y ago
      Google doesn't force you to do AdSense an all pages. And Analytics, well, you can also skip on sensitive pages.
gpm 7y ago
4. Power users and nerds need to upstream the privacy improvements to everyone. - To do so you need to avoid making user experience worse
- wlesieutre 7y ago
  Apple's work on tracking protection in Safari are a great step for this. Normalizing ITP across the whole Mac / iOS / iPad OS userbase means that sites have to accept it or block a huge number of normal users.
hombre_fatal 7y ago
Yeah, you can't really talk about downsides of Recaptcha/Cloudflare without also acknowledging the extreme amount of malicious actors and abuse on the internet. We're in a "this is why we can't have nice things" predicament and you have malicious actors to thank for that, yet most people on HN only seem capable of attacking the few affordable solutions to that problem. I'm even down with the theory that Cloudflare is a US government outfit, that's the only way I can wrap my head around such a generous free tier. But at what point does it worry you that the internet has so many fundamental issues that people willingly centralize behind such a large behemoth? How many options do I have when a kid is holding my forum hostage with a $5 booter service? It's easy to shit on everything. Let's hear some real solutions.
- deleted 7y ago
  [deleted]
- thaumasiotes 7y ago
  > How are you going to talk about the downsides of Recaptcha/Cloudflare without also acknowledging the extreme amount of malicious actors and abuse on the internet? This is acknowledged by the original question >> Also, how would your implementation differ to solve this issue?
  - hombre_fatal 7y ago
    To be clear, I agree with their comment and tried to double down on their point that it's a tough nut to crack. I improved my opening line.
- z3t4 7y ago
  For caching: Learn how to code. If your web page dies when there are only two visitors, then that's on you. DDoS attack: If possible, the easiest solution is to just swallow the traffic. If that doesn't work you want to block all networks that allows IP spoofing. Then it's a wack-a-mole game. And if you have the resources, use any-cast and many co-locations. Or ask your ISP for help. Hiding your server: Use onion address via TOR network. SSL certificate: Use Letsencrypt Edge SSL/DNS/CDN: Use a fast web server or proxy, like Nginx. With Cloudflare the connection to the Edge server might be faster, but time to first byte (on your site) often slower. So you get better bang for the buck by optimizing on your end. Note that DNS by itself already have edge caching out of the box, for free! eg. if a user looks up your domain, it will be cached both at their ISP and LAN. So you don't need Cloudflare for DNS.
hwbehrens 7y ago
> Two steps backwards in every conceivable way. The giants gain more invisible power and powerusers suffer decreased productivity/privacy. Not going to happen. I agree with the first two sentences, but disagree with the third. I believe that this state is actually the intended end goal. Previously, for many years, I browsed the web with Javascript disabled. At the time, this had very little impact on my browsing experience; perhaps some of the layout would be broken, but not in a way that would interfere with the functionality or content of the site. Nowadays, not only is this totally impossible, blocking even a subset of a site's JS (such as through uMatrix) is trial and error to get the site to load at all, or to do simple tasks like click on a "login" button. With Google's plan to "phase out" cookies [0], I expect the web to become even more opaque and difficult to modify "on the fly" -- that is, on the user's local machine prior to displaying the content. In particular, this will affect ad and tracker blocking the most, as the pain from effective ad blockers starts to bite harder and harder. So, when you say "The giants gain more invisible power", that is true and desireable from their perspective, and since they write the code that actually underpins most web browsers, why wouldn't they? When you say, "powerusers suffer decreates productivity/privacy", yes, that's absolutely true. Why would they care? It's such a small fraction of their business. Some users will go to more and more extremes to preserve their privacy, eventually accessing only some small subset of sites from an esoteric Kali-derived distro, and others will capitulate and shift their behavior back to the herd. In the end, the giants still win. [0]: https://www.inc.com/jason-aten/google-says-chrome-will-end-support-for-third-party-cookies-that-track-you-thats-not-all-good-news.html https://www.inc.com/jason-aten/google-says-chrome-will-end-s...
- Liquix 7y ago
  Great points, I absolutely agree with everything you've said. The 'not going to happen' is less a prediction of the future and more a reflection of my personal stubbornness/frustration regarding the direction things are headed (increased 'opacity' as you put it).
- isodude 7y ago
  Like always we find another venue where the giants haven't their foot yet. Or the malicous users does not find a huge interest in. Rebooting the web into something else is still possible and something that will eventually happen when enough people are too tired about the current state. This battle has been lost long time ago.
sandov 7y ago
4. Boycott Cloudflare.
- lucb1e 7y ago
  Isn't that #1? Or do you mean more like what OP is experiencing by being banned? GP already mentioned why #1 is not a solution (which I see the same way) and OP made it quite clear that not visiting CF sites isn't quite working either.
RileyJames 7y ago
4. Outsource. Use a ReCaptcha/Cloudflare filling service (usually just someone else manually typing these in). I won’t link, but search “ReCaptcha solver” and you’ll find plenty. It highlights just how broken the system is. It doesn’t stop determined spammers/devs, until the value of the task is lower than the cost to solve. Considering it’s 50c USD per 1000....
- xur17 7y ago
  I've seriously considered doing exactly this. I've solved numerous challenges for Google this week alone.
- jabart 7y ago
  Everyone keeps bringing this up but unless you have something of monetary value on the other end, this won't happen. ReCaptcha and a few if statements has stopped all contact us form spam on our site. Same for other sites I help to manage, no one is paying 50 cents per a thousand contact us spam messages.
  - ficklepickle 7y ago
    I found all my contact form spam was being sent superhumanly fast. Well under 10 seconds from initial page load. A user can't make it to the form and type a meaningful message that fast. Adding a short timeout eliminated my contact form spam. I also only allow JSON on the back end, so they must execute JS to even have a shot. This has allowed me to avoid blocking TOR exit nodes... So far anyway.
A4ET8a8uTh0 7y ago
I don't see myself as a poweruser, but it would be very hard for me to give up on my current setup. I honestly do not understand how my wife deal with it. Even with piholed wifi it is a horrifying experience..
heyyyouu 7y ago
Aren't there configuration options/levels that can be set within CloudFlare to mitigate these issues? EDIT: Another user posted this below, answering my question: sgtfrankieboy 1 hour ago | undown [-] In CloudFlare go to "Firewall" and then click Settings on the right. Here you can set the Security Level and if you want to use Browser Integrity Checks among other thing
- llama052 7y ago
  Yup there sure is, but the Cloudflare hate bandwagon has started. This is more on the site-owner than Cloudflare in my opinion.
mirimir 7y ago
My solution is connecting via nested VPN chains, working in VMs, and compartmentalizing stuff using multiple personas. Reputable VPN services do a good job of keeping their IPs off blocklists. Occasionally I'll get blocked, because some jerk has been abusing the VPN server that I'm exiting from. But if it doesn't resolve promptly, I just switch to a different exit server. So I only use this VM, and this VPN exit, as Mirimir. And given that, I don't go out of my way to prevent tracking. Not enough, anyway, to trigger blocking. Because I don't really care if everything that Mirimir does gets linked. Indeed, I pretty much always use "Mirimir" as my username, or sometimes "Dimi" or whatever. If I don't want stuff linked, I use a different persona in a different VM, using a different VPN chain. Or that via Tor using Whonix.
mhasbini 7y ago
> Launch a competing product that accomplishes the same thing. Disclaimer: I was part of hCaptcha team. https://hcaptcha.com/ https://hcaptcha.com/ is competing with ReCaptcha. It's a drop-in replacement for ReCaptcha. It's privacy focused (supports privacy pass), and is fair: webmasters get a cut for each captcha that is solved correctly (they can choose to directly donate it to a charity of their choice), hCaptcha get a cut for running the service and a customer will get their images/data labeled.
- nemoniac 7y ago
  An alternative? This service uses both Cloudfare and googleapis. What's that all about?
  - deleted 7y ago
    [deleted]
kragen 7y ago
4. Mirror the contents of this Dark Web 3.0 on a Light Web 3.0 accessed using privacy-protecting technologies like Tor to make sure it remains accessible. Obviously this won't help for logging into Twitter and your Fecebutt account but it should be fine for the nginx documentation. The underlying incentive here is that centralized websites are slow and vulnerable to DDoS. Massively mirroring their content is the solution, and it's what Cloudflare does. Let's do it in a way that protects human rights rather than taking them away.
errnoh 7y ago
This is an interesting subject indeed. Even though both of them are in the "try to grab as big portion of internet traffic as possible" business I wouldn't compare them that easily. Cloudflare is actually rather good in doing what they do (DDoS side of things). They rarely break normal internet use, the only time when that happens is when a site is put into the "I'm under attack" mode that forces browser to do javascript proof. They do get huge amounts of traffic information though, but that is pretty much required to their core business (DDoS prevention, not the tinfoil kind) Google/ReCaptcha is another thing. I have hard time understanding any reason to put captcha on any site that a normal incremental delay between login tries & ban sources that keep doing that for too long wouldn't already prevent. They're getting traffic data and ML training data and neither one is required for the thing captcha is trying to solve. Sites are just feeding their business and captcha is actually making the internet worse place for humans. (Captcha requirement for things like posting on a discussion could be handled by simple spam/bot detection, captcha is just overkill)
jakecopp 7y ago
I'd love to use an open source/more beneficial to society version of ReCaptcha - say validating OpenStreetMap data/project Gutenberg/something else. Maybe this already exists and I don't know about it! Would sites then move to this and reduce the lock in and inflexibility with ReCaptcha?
jpalomaki 7y ago
I think the long term solution could be in making the network providers and ISPs more responsible for the malicious traffic originating from their networks. For example the botnet traffic is best stopped at the origin. If there was some pressure for the service providers, I'm fairly sure they could do more to detect subscriptions with compromised devices and take appropriate actions. Actions can include educating the users and if necessary, blocking the subscription until problems are fixed. While this certainly would not immediately cover the whole world, it would be a start. On the website level you could then treat traffic from networks that have agreed to cut malicious traffic in different way.
GoblinSlayer 7y ago
ReCaptcha works because any captcha works. https://kevv.net/you-probably-dont-need-recaptcha/ https://kevv.net/you-probably-dont-need-recaptcha/ https://blog.codinghorror.com/captcha-effectiveness/ https://blog.codinghorror.com/captcha-effectiveness/

by Liquix 7y ago

sturgill 7y ago

I’m a fan of Recaptcha v3. There are many actions where you can ask for some additional input in a non-puzzle way. Simple example is sending a confirmation email before signup when the score is below a certain threshold. Because I freaking hate those captcha puzzles...

rstupek 7y ago
Except v3 has to be on every page of the site and that gives Google a full view of all traffic on the site
- WillPostForFood 7y ago
  So does Google Analytics, so does AdSense. ReCaptcha V3 is probably the least used of the three.
  - catalogia 7y ago
    Blocking GA and AdSense doesn't render websites unusable.
  - ddalex 7y ago
    Google doesn't force you to do AdSense an all pages. And Analytics, well, you can also skip on sensitive pages.

gpm 7y ago

4. Power users and nerds need to upstream the privacy improvements to everyone. - To do so you need to avoid making user experience worse

wlesieutre 7y ago
Apple's work on tracking protection in Safari are a great step for this. Normalizing ITP across the whole Mac / iOS / iPad OS userbase means that sites have to accept it or block a huge number of normal users.

hombre_fatal 7y ago

Yeah, you can't really talk about downsides of Recaptcha/Cloudflare without also acknowledging the extreme amount of malicious actors and abuse on the internet. We're in a "this is why we can't have nice things" predicament and you have malicious actors to thank for that, yet most people on HN only seem capable of attacking the few affordable solutions to that problem. I'm even down with the theory that Cloudflare is a US government outfit, that's the only way I can wrap my head around such a generous free tier. But at what point does it worry you that the internet has so many fundamental issues that people willingly centralize behind such a large behemoth? How many options do I have when a kid is holding my forum hostage with a $5 booter service? It's easy to shit on everything. Let's hear some real solutions.

deleted 7y ago
[deleted]
thaumasiotes 7y ago
> How are you going to talk about the downsides of Recaptcha/Cloudflare without also acknowledging the extreme amount of malicious actors and abuse on the internet? This is acknowledged by the original question >> Also, how would your implementation differ to solve this issue?
- hombre_fatal 7y ago
  To be clear, I agree with their comment and tried to double down on their point that it's a tough nut to crack. I improved my opening line.
z3t4 7y ago
For caching: Learn how to code. If your web page dies when there are only two visitors, then that's on you. DDoS attack: If possible, the easiest solution is to just swallow the traffic. If that doesn't work you want to block all networks that allows IP spoofing. Then it's a wack-a-mole game. And if you have the resources, use any-cast and many co-locations. Or ask your ISP for help. Hiding your server: Use onion address via TOR network. SSL certificate: Use Letsencrypt Edge SSL/DNS/CDN: Use a fast web server or proxy, like Nginx. With Cloudflare the connection to the Edge server might be faster, but time to first byte (on your site) often slower. So you get better bang for the buck by optimizing on your end. Note that DNS by itself already have edge caching out of the box, for free! eg. if a user looks up your domain, it will be cached both at their ISP and LAN. So you don't need Cloudflare for DNS.

hwbehrens 7y ago

> Two steps backwards in every conceivable way. The giants gain more invisible power and powerusers suffer decreased productivity/privacy. Not going to happen. I agree with the first two sentences, but disagree with the third. I believe that this state is actually the intended end goal. Previously, for many years, I browsed the web with Javascript disabled. At the time, this had very little impact on my browsing experience; perhaps some of the layout would be broken, but not in a way that would interfere with the functionality or content of the site. Nowadays, not only is this totally impossible, blocking even a subset of a site's JS (such as through uMatrix) is trial and error to get the site to load at all, or to do simple tasks like click on a "login" button. With Google's plan to "phase out" cookies [0], I expect the web to become even more opaque and difficult to modify "on the fly" -- that is, on the user's local machine prior to displaying the content. In particular, this will affect ad and tracker blocking the most, as the pain from effective ad blockers starts to bite harder and harder. So, when you say "The giants gain more invisible power", that is true and desireable from their perspective, and since they write the code that actually underpins most web browsers, why wouldn't they? When you say, "powerusers suffer decreates productivity/privacy", yes, that's absolutely true. Why would they care? It's such a small fraction of their business. Some users will go to more and more extremes to preserve their privacy, eventually accessing only some small subset of sites from an esoteric Kali-derived distro, and others will capitulate and shift their behavior back to the herd. In the end, the giants still win. [0]: https://www.inc.com/jason-aten/google-says-chrome-will-end-support-for-third-party-cookies-that-track-you-thats-not-all-good-news.html https://www.inc.com/jason-aten/google-says-chrome-will-end-s...

Liquix 7y ago
Great points, I absolutely agree with everything you've said. The 'not going to happen' is less a prediction of the future and more a reflection of my personal stubbornness/frustration regarding the direction things are headed (increased 'opacity' as you put it).
isodude 7y ago
Like always we find another venue where the giants haven't their foot yet. Or the malicous users does not find a huge interest in. Rebooting the web into something else is still possible and something that will eventually happen when enough people are too tired about the current state. This battle has been lost long time ago.

sandov 7y ago

4. Boycott Cloudflare.

lucb1e 7y ago
Isn't that #1? Or do you mean more like what OP is experiencing by being banned? GP already mentioned why #1 is not a solution (which I see the same way) and OP made it quite clear that not visiting CF sites isn't quite working either.

RileyJames 7y ago

4. Outsource. Use a ReCaptcha/Cloudflare filling service (usually just someone else manually typing these in). I won’t link, but search “ReCaptcha solver” and you’ll find plenty. It highlights just how broken the system is. It doesn’t stop determined spammers/devs, until the value of the task is lower than the cost to solve. Considering it’s 50c USD per 1000....

xur17 7y ago
I've seriously considered doing exactly this. I've solved numerous challenges for Google this week alone.
jabart 7y ago
Everyone keeps bringing this up but unless you have something of monetary value on the other end, this won't happen. ReCaptcha and a few if statements has stopped all contact us form spam on our site. Same for other sites I help to manage, no one is paying 50 cents per a thousand contact us spam messages.
- ficklepickle 7y ago
  I found all my contact form spam was being sent superhumanly fast. Well under 10 seconds from initial page load. A user can't make it to the form and type a meaningful message that fast. Adding a short timeout eliminated my contact form spam. I also only allow JSON on the back end, so they must execute JS to even have a shot. This has allowed me to avoid blocking TOR exit nodes... So far anyway.

A4ET8a8uTh0 7y ago

I don't see myself as a poweruser, but it would be very hard for me to give up on my current setup. I honestly do not understand how my wife deal with it. Even with piholed wifi it is a horrifying experience..

heyyyouu 7y ago

Aren't there configuration options/levels that can be set within CloudFlare to mitigate these issues? EDIT: Another user posted this below, answering my question: sgtfrankieboy 1 hour ago | undown [-] In CloudFlare go to "Firewall" and then click Settings on the right. Here you can set the Security Level and if you want to use Browser Integrity Checks among other thing

llama052 7y ago
Yup there sure is, but the Cloudflare hate bandwagon has started. This is more on the site-owner than Cloudflare in my opinion.

mirimir 7y ago

My solution is connecting via nested VPN chains, working in VMs, and compartmentalizing stuff using multiple personas. Reputable VPN services do a good job of keeping their IPs off blocklists. Occasionally I'll get blocked, because some jerk has been abusing the VPN server that I'm exiting from. But if it doesn't resolve promptly, I just switch to a different exit server. So I only use this VM, and this VPN exit, as Mirimir. And given that, I don't go out of my way to prevent tracking. Not enough, anyway, to trigger blocking. Because I don't really care if everything that Mirimir does gets linked. Indeed, I pretty much always use "Mirimir" as my username, or sometimes "Dimi" or whatever. If I don't want stuff linked, I use a different persona in a different VM, using a different VPN chain. Or that via Tor using Whonix.

mhasbini 7y ago

> Launch a competing product that accomplishes the same thing. Disclaimer: I was part of hCaptcha team. https://hcaptcha.com/ https://hcaptcha.com/ is competing with ReCaptcha. It's a drop-in replacement for ReCaptcha. It's privacy focused (supports privacy pass), and is fair: webmasters get a cut for each captcha that is solved correctly (they can choose to directly donate it to a charity of their choice), hCaptcha get a cut for running the service and a customer will get their images/data labeled.

nemoniac 7y ago
An alternative? This service uses both Cloudfare and googleapis. What's that all about?
- deleted 7y ago
  [deleted]

kragen 7y ago

4. Mirror the contents of this Dark Web 3.0 on a Light Web 3.0 accessed using privacy-protecting technologies like Tor to make sure it remains accessible. Obviously this won't help for logging into Twitter and your Fecebutt account but it should be fine for the nginx documentation. The underlying incentive here is that centralized websites are slow and vulnerable to DDoS. Massively mirroring their content is the solution, and it's what Cloudflare does. Let's do it in a way that protects human rights rather than taking them away.

errnoh 7y ago

This is an interesting subject indeed. Even though both of them are in the "try to grab as big portion of internet traffic as possible" business I wouldn't compare them that easily. Cloudflare is actually rather good in doing what they do (DDoS side of things). They rarely break normal internet use, the only time when that happens is when a site is put into the "I'm under attack" mode that forces browser to do javascript proof. They do get huge amounts of traffic information though, but that is pretty much required to their core business (DDoS prevention, not the tinfoil kind) Google/ReCaptcha is another thing. I have hard time understanding any reason to put captcha on any site that a normal incremental delay between login tries & ban sources that keep doing that for too long wouldn't already prevent. They're getting traffic data and ML training data and neither one is required for the thing captcha is trying to solve. Sites are just feeding their business and captcha is actually making the internet worse place for humans. (Captcha requirement for things like posting on a discussion could be handled by simple spam/bot detection, captcha is just overkill)

jakecopp 7y ago

I'd love to use an open source/more beneficial to society version of ReCaptcha - say validating OpenStreetMap data/project Gutenberg/something else. Maybe this already exists and I don't know about it! Would sites then move to this and reduce the lock in and inflexibility with ReCaptcha?

jpalomaki 7y ago

I think the long term solution could be in making the network providers and ISPs more responsible for the malicious traffic originating from their networks. For example the botnet traffic is best stopped at the origin. If there was some pressure for the service providers, I'm fairly sure they could do more to detect subscriptions with compromised devices and take appropriate actions. Actions can include educating the users and if necessary, blocking the subscription until problems are fixed. While this certainly would not immediately cover the whole world, it would be a start. On the website level you could then treat traffic from networks that have agreed to cut malicious traffic in different way.

GoblinSlayer 7y ago

ReCaptcha works because any captcha works. https://kevv.net/you-probably-dont-need-recaptcha/ https://kevv.net/you-probably-dont-need-recaptcha/ https://blog.codinghorror.com/captcha-effectiveness/ https://blog.codinghorror.com/captcha-effectiveness/