15 ms·

Well into the second day of Cloudflare’s blockade of my home internet connection, Google Search also began blocking requests. It required me to resolve a CAPTCH

Well into the second day of Cloudflare’s blockade of my home internet connection, Google Search also began blocking requests. It required me to resolve a CAPTCHA challenge for every other search. This luckily only lasted a day. Cloudflare shares IP reputation data with partners like Google, coordinated through a program called the Bandwidth Alliance. So, my original offense might not even have been against Cloudflare. It might have received the reputation data from a partner, and it just propagated through the Bandwidth Alliance network. That's not what Bandwidth Alliance is at all. It's about reducing or eliminating egress fees between a cloud provider and Cloudflare. Not sure where the idea that it's about sharing IP reputation data comes from. https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/ So, if Google Search started showing a CAPTCHA that's not Cloudflare.

pilif 4y ago
Yep. That paragraph made me pause and consider that maybe OP is the victim of some compromised device running on their network. If two independent sites believe you are a bot, you or something at your address just might be.
O__________O 4y ago
They do have a threat score https://developers.cloudflare.com/firewall/recipes/block-ip-reputation/ https://developers.cloudflare.com/firewall/recipes/block-ip-... I was surprised to learn Cloudflare was born out of Project Honeypot, so I am guessing Cloudflare does share data with them: https://www.projecthoneypot.org/cloudflare_beta.html https://www.projecthoneypot.org/cloudflare_beta.html
- deleted 4y ago
  [deleted]
- elcomet 4y ago
  FYI you're responding to the cloudflare CTO
  - trasz 4y ago
    It’s naive to assume Cloudflare CTO would not be lying if beneficial to him or Cloudflare.
    - Veen 4y ago
      It's even more naive to assume Cloudflare's CTO would tell lies that can be trivially shown to be untrue.
      - pessimizer 4y ago
      - trasz 4y ago
        How would you show they are untrue? Ask? :-D
    - elcomet 4y ago
      I don't assume anything. The previous comment was just trying to teach something about cloudflare to its CTO
    - nemothekid 4y ago
      I wonder if HN posters have ever held a job before. Can you explain why it's beneficial for Cloudflare to block legitimate users? Why is the simplest explanation "Cloudflare just hates this one user in particular?"
      - lmm 4y ago
        Well, apparently they scared this user into installing their browser extension, so it sounds like this incident was a win for them.
        Sebb767 4y ago
        And they also got them to write a blog post, giving massively negative press on HN. I doubt his PII (assuming they collect it) is worth the trouble this thread is causing.
cvwright 4y ago
You block this guy from the internet for a week —- for no apparent reason —- and then you come in here with a nitpick about how another related system works? Really?
- judge2020 4y ago
  The point is that Cloudflare does not beam IP reputation data to Google. If Google and CF are blocking this IP separately, what's the chance there's some malicious device or hacked IoT device on the network, participating in DDOS attacks or unauthorized vulnerability scanning of random websites?
  - zinekeller 4y ago
    Yeah, if for example Spamhaus (which both Cloudflare and Google consult) has detected that a subnet is bad then that could be the cause. Still, it doesn't excuse Cloudflare that there's no redress if you are caught on a block or even a clue on what you can do to reduce it (especially that Spamhaus do have redress procedures).
  - cvwright 4y ago
    Fair point
  - pessimizer 4y ago
    According to another comment, it's a wrong point: https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/ > Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost.
    - jgrahamc 4y ago
      Yeah. I'm looking into that.
phantom_of_cato 4y ago
But that's beside the main point. You guys are essentially the "single point of failure" for half the internet. [1] Being competent and smart doesn't really help too much, as demonstrated by how you guys had to give in to the pressure to censor recently. [1]: https://easydns.com/blog/2020/07/20/turns-out-half-the-internet-has-a-single-point-of-failure-called-cloudflare/ https://easydns.com/blog/2020/07/20/turns-out-half-the-inter...
TakeBlaster16 4y ago
Can you acknowledge the main point of the article? What should someone do if they find themselves misclassified by Cloudflare's systems?
- mh- 4y ago
  (not the parent commenter) That person should start with the assumption they haven't been misclassified and eliminate the possibility that a device on their network is compromised.
  - d2wa 4y ago
    (Author here.) That’s missing from the article. But I have logs of the network. There’s nothing out of the ordinary. “I don’t know what I did wrong,” as I started the article, means “I’ve checked logs and such and there’s no indication of anything wrong on my end.”
  - JohnFen 4y ago
    A task that would be made much easier and less likely to miss something if the affected person had some indication as to what the problem was.
    - buildbot 4y ago
      Devil's advocate - would it not then be pretty easy to engineer malicious bots to avoid detection?
      - JohnFen 4y ago
        Depends on the level of detail provided. That much detail isn't necessary in order to provide a helpful pointer to innocent bystanders.
  - Ferret7446 4y ago
    Do you expect the average user to know how to "eliminate the possibility that a device on their network is compromised"? That is untenable.
tomxor 4y ago
FYI, this guy is far from alone, your "protection" has given me a lot of grief over the past few years, particularly on highly NATed mobile networks. I've been gradually removing cloudflare based CDNs from services I develop and control because I don't want my users being arbitrarily discriminated against. There was a good article posted on HN recently titled "The ideal level of fraud is non-zero" which I think is highly relevant here... In essence any mechanism employed to prevent illegitimate use comes with a negative cost to legitimate users, if that cost is too high it defeats the purpose. i.e what's the point in a website that is completely immune to a botnet and also cannot be accessed by anyone else? unplugging the ethernet cable also effectively protects against botnets. More subtly the cost of outright rejecting some legitimate users is usually not worth the savings of rejecting 100% of illegitimate ones. I think Cloudflare's service has it the wrong way around: it currently accept blocking legitimate users far too easily, that is not an acceptable cost; whereas you should be letting a higher level of bots through to avoid pissing off legitimate users - if it's not obviously a DDoS, it's probably worth the bandwidth cost. Consider the bigger picture, if you save a slither of a penny by blocking a bot, but also end up blocking or seriously inconveniencing 10 real users... is it worth it.
- dmix 4y ago
  Cloudflare just isn't worth the tradeoffs: the risks associated with their centralization, how they made Tor basically unusable on non-onion sites, the lack of transparency when content-moderating the internet, etc. The space is in need of solid competitors to break the stranglehold they have on the internet. Whether it's the right combination of services, documentation, etc.
  - andrewnyr 4y ago
    there are many solid competitors: Amazon, Fastly, Akamai, Imperva to name a few
    - wahnfrieden 4y ago
      Bunny
throwawayays 4y ago
The tone of this reply is a bit shit from a PR perspective. How about _also_ pointing to a knowledge base article for how an end user could go about working out what network activity from their IP might be flagging Cloudflare’s systems?
d2wa 4y ago
> That's not what Bandwidth Alliance is at all. It's about reducing or eliminating egress fees between a cloud provider and Cloudflare. Not sure where the idea that it's about sharing IP reputation data comes from. It comes from the Cloudflare blog. https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/ There’s a support page about it too. https://developers.cloudflare.com/bots/get-started/free/ https://developers.cloudflare.com/bots/get-started/free/
- jgrahamc 4y ago
  I need to look into that. Thanks for pointing it out. I had totally forgotten about that post. Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.
  - d2wa 4y ago
    I edited and added a second link to a support page that mentions it too.
    - jgrahamc 4y ago
      Thanks. I'm talking with the team. Edit: see comment above.
  - d2wa 4y ago
    Dear John. What am I — as a normal human being/end-user — supposed to do in this situation? People can’t do anything without any information about why they’re blocked. Who do you contact? Where do you go? What to do? The challenge page doesn’t help the end user understand why this is happening to them. It’s okay if you only see it for two seconds. But the page stays on screen for over a minute. When this happens for every website — what do you do? You’d be furious if this had happen to you. I’m just trying to read my online comics and lookup some stuff about some interests and hobbies. It reduced my quality of life/sanity for a week. The last two days, I started worrying that this was going to be the new normal. I even looked into swapping ISP to get a new IP address. PS: I love all the innovation and engineering stuff you guys regularly share on the Cloudflare blog. It’s [almost] always an interesting read. Even though I’m no fan of the massive centralization your company has caused.
deleted 4y ago
[deleted]
xani_ 4y ago
> Not sure where the idea that it's about sharing IP reputation data comes from. Probably from scam called mail blacklists
plumeria 4y ago
It is interesting that the Bandwidth Alliance partners list shows pretty much every big cloud provider except AWS and Akamai [0] [0] https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/
bogomipz 4y ago
>"Not sure where the idea that it's about sharing IP reputation data comes from." One source of that would be a blog post on your company's website that was actually authored by you! Point 2 below: >"Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost. [1] So it's not such a far-fetched notion is it? [1] https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/
3np 4y ago
What happened to PrivacyPass? It seems to have stopped working completely when connecting over Tor several months back. I say this from having spent several hours trying to get it work on multiple devices with different OS/client software (chromium/FF), both with the store versions and bundling the extension from source. We did have it working mostly fine for some time back in 2021 but haven't been able to since. There are multiple open issues reporting this on the GH repo with no real follow-up from maintainers apart from maybe a "should be fixed, open again if still an issue". ie https://github.com/privacypass/challenge-bypass-extension/issues/337 https://github.com/privacypass/challenge-bypass-extension/is...

pilif 4y ago

Yep. That paragraph made me pause and consider that maybe OP is the victim of some compromised device running on their network. If two independent sites believe you are a bot, you or something at your address just might be.

O__________O 4y ago

They do have a threat score https://developers.cloudflare.com/firewall/recipes/block-ip-reputation/ https://developers.cloudflare.com/firewall/recipes/block-ip-... I was surprised to learn Cloudflare was born out of Project Honeypot, so I am guessing Cloudflare does share data with them: https://www.projecthoneypot.org/cloudflare_beta.html https://www.projecthoneypot.org/cloudflare_beta.html

deleted 4y ago
[deleted]
elcomet 4y ago
FYI you're responding to the cloudflare CTO
- trasz 4y ago
  It’s naive to assume Cloudflare CTO would not be lying if beneficial to him or Cloudflare.
  - Veen 4y ago
    It's even more naive to assume Cloudflare's CTO would tell lies that can be trivially shown to be untrue.
    - pessimizer 4y ago
    - trasz 4y ago
      How would you show they are untrue? Ask? :-D
  - elcomet 4y ago
    I don't assume anything. The previous comment was just trying to teach something about cloudflare to its CTO
  - nemothekid 4y ago
    I wonder if HN posters have ever held a job before. Can you explain why it's beneficial for Cloudflare to block legitimate users? Why is the simplest explanation "Cloudflare just hates this one user in particular?"
    - lmm 4y ago
      Well, apparently they scared this user into installing their browser extension, so it sounds like this incident was a win for them.
      - Sebb767 4y ago
        And they also got them to write a blog post, giving massively negative press on HN. I doubt his PII (assuming they collect it) is worth the trouble this thread is causing.

cvwright 4y ago

You block this guy from the internet for a week —- for no apparent reason —- and then you come in here with a nitpick about how another related system works? Really?

judge2020 4y ago
The point is that Cloudflare does not beam IP reputation data to Google. If Google and CF are blocking this IP separately, what's the chance there's some malicious device or hacked IoT device on the network, participating in DDOS attacks or unauthorized vulnerability scanning of random websites?
- zinekeller 4y ago
  Yeah, if for example Spamhaus (which both Cloudflare and Google consult) has detected that a subnet is bad then that could be the cause. Still, it doesn't excuse Cloudflare that there's no redress if you are caught on a block or even a clue on what you can do to reduce it (especially that Spamhaus do have redress procedures).
- cvwright 4y ago
  Fair point
- pessimizer 4y ago
  According to another comment, it's a wrong point: https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/ > Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost.
  - jgrahamc 4y ago
    Yeah. I'm looking into that.

phantom_of_cato 4y ago

But that's beside the main point. You guys are essentially the "single point of failure" for half the internet. [1] Being competent and smart doesn't really help too much, as demonstrated by how you guys had to give in to the pressure to censor recently. [1]: https://easydns.com/blog/2020/07/20/turns-out-half-the-internet-has-a-single-point-of-failure-called-cloudflare/ https://easydns.com/blog/2020/07/20/turns-out-half-the-inter...

TakeBlaster16 4y ago

Can you acknowledge the main point of the article? What should someone do if they find themselves misclassified by Cloudflare's systems?

mh- 4y ago
(not the parent commenter) That person should start with the assumption they haven't been misclassified and eliminate the possibility that a device on their network is compromised.
- d2wa 4y ago
  (Author here.) That’s missing from the article. But I have logs of the network. There’s nothing out of the ordinary. “I don’t know what I did wrong,” as I started the article, means “I’ve checked logs and such and there’s no indication of anything wrong on my end.”
- JohnFen 4y ago
  A task that would be made much easier and less likely to miss something if the affected person had some indication as to what the problem was.
  - buildbot 4y ago
    Devil's advocate - would it not then be pretty easy to engineer malicious bots to avoid detection?
    - JohnFen 4y ago
      Depends on the level of detail provided. That much detail isn't necessary in order to provide a helpful pointer to innocent bystanders.
- Ferret7446 4y ago
  Do you expect the average user to know how to "eliminate the possibility that a device on their network is compromised"? That is untenable.

tomxor 4y ago

FYI, this guy is far from alone, your "protection" has given me a lot of grief over the past few years, particularly on highly NATed mobile networks. I've been gradually removing cloudflare based CDNs from services I develop and control because I don't want my users being arbitrarily discriminated against. There was a good article posted on HN recently titled "The ideal level of fraud is non-zero" which I think is highly relevant here... In essence any mechanism employed to prevent illegitimate use comes with a negative cost to legitimate users, if that cost is too high it defeats the purpose. i.e what's the point in a website that is completely immune to a botnet and also cannot be accessed by anyone else? unplugging the ethernet cable also effectively protects against botnets. More subtly the cost of outright rejecting some legitimate users is usually not worth the savings of rejecting 100% of illegitimate ones. I think Cloudflare's service has it the wrong way around: it currently accept blocking legitimate users far too easily, that is not an acceptable cost; whereas you should be letting a higher level of bots through to avoid pissing off legitimate users - if it's not obviously a DDoS, it's probably worth the bandwidth cost. Consider the bigger picture, if you save a slither of a penny by blocking a bot, but also end up blocking or seriously inconveniencing 10 real users... is it worth it.

dmix 4y ago
Cloudflare just isn't worth the tradeoffs: the risks associated with their centralization, how they made Tor basically unusable on non-onion sites, the lack of transparency when content-moderating the internet, etc. The space is in need of solid competitors to break the stranglehold they have on the internet. Whether it's the right combination of services, documentation, etc.
- andrewnyr 4y ago
  there are many solid competitors: Amazon, Fastly, Akamai, Imperva to name a few
  - wahnfrieden 4y ago
    Bunny

throwawayays 4y ago

The tone of this reply is a bit shit from a PR perspective. How about _also_ pointing to a knowledge base article for how an end user could go about working out what network activity from their IP might be flagging Cloudflare’s systems?

d2wa 4y ago

> That's not what Bandwidth Alliance is at all. It's about reducing or eliminating egress fees between a cloud provider and Cloudflare. Not sure where the idea that it's about sharing IP reputation data comes from. It comes from the Cloudflare blog. https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/ There’s a support page about it too. https://developers.cloudflare.com/bots/get-started/free/ https://developers.cloudflare.com/bots/get-started/free/

jgrahamc 4y ago
I need to look into that. Thanks for pointing it out. I had totally forgotten about that post. Edit: team tells me this idea never got off the ground. Did talk with some potential partners (which did NOT include Google) but didn’t happen. So if Google was throwing CAPTCHAs it wasn’t because of our IP reputation.
- d2wa 4y ago
  I edited and added a second link to a support page that mentions it too.
  - jgrahamc 4y ago
    Thanks. I'm talking with the team. Edit: see comment above.
- d2wa 4y ago
  Dear John. What am I — as a normal human being/end-user — supposed to do in this situation? People can’t do anything without any information about why they’re blocked. Who do you contact? Where do you go? What to do? The challenge page doesn’t help the end user understand why this is happening to them. It’s okay if you only see it for two seconds. But the page stays on screen for over a minute. When this happens for every website — what do you do? You’d be furious if this had happen to you. I’m just trying to read my online comics and lookup some stuff about some interests and hobbies. It reduced my quality of life/sanity for a week. The last two days, I started worrying that this was going to be the new normal. I even looked into swapping ISP to get a new IP address. PS: I love all the innovation and engineering stuff you guys regularly share on the Cloudflare blog. It’s [almost] always an interesting read. Even though I’m no fan of the massive centralization your company has caused.

deleted 4y ago

[deleted]

xani_ 4y ago

> Not sure where the idea that it's about sharing IP reputation data comes from. Probably from scam called mail blacklists

plumeria 4y ago

It is interesting that the Bandwidth Alliance partners list shows pretty much every big cloud provider except AWS and Akamai [0] [0] https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/

bogomipz 4y ago

>"Not sure where the idea that it's about sharing IP reputation data comes from." One source of that would be a blog post on your company's website that was actually authored by you! Point 2 below: >"Once enabled, when we detect a bad bot, we will do three things: (1) we’re going to disincentivize the bot maker economically by tarpitting them, including requiring them to solve a computationally intensive challenge that will require more of their bot’s CPU; (2) for Bandwidth Alliance partners, we’re going to hand the IP of the bot to the partner and get the bot kicked offline; and (3) we’re going to plant trees to make up for the bot’s carbon cost. [1] So it's not such a far-fetched notion is it? [1] https://blog.cloudflare.com/cleaning-up-bad-bots/ https://blog.cloudflare.com/cleaning-up-bad-bots/

3np 4y ago

What happened to PrivacyPass? It seems to have stopped working completely when connecting over Tor several months back. I say this from having spent several hours trying to get it work on multiple devices with different OS/client software (chromium/FF), both with the store versions and bundling the extension from source. We did have it working mostly fine for some time back in 2021 but haven't been able to since. There are multiple open issues reporting this on the GH repo with no real follow-up from maintainers apart from maybe a "should be fixed, open again if still an issue". ie https://github.com/privacypass/challenge-bypass-extension/issues/337 https://github.com/privacypass/challenge-bypass-extension/is...