37 ms·

Blocked by Cloudflare

632 points by jrhawley 3y ago

warrenm 3y ago
[flagged]
kozhevnikov 3y ago
> I temporarily disabled extensions. I opened a private browsing window. FYI When using Chrome, incognito window carries a lot of baggage. For issues like this use Guest profile as it doesn't include extensions, caches, storage, etc. Optionally do a Google search first to seed it with cookies.
- jsnell 3y ago
  Doing a Google search will not populate any cookies Cloudflare could access.
  - kozhevnikov 3y ago
    Of course not, but it is less likely to trigger manual recaptcha on the site you're trying to visit in Guest mode.
    - Operyl 3y ago
      Cloudflare does not use Google's reCAPTCHA anymore, and hasn't for some time.
- fireflash38 3y ago
  Mind expanding on what you mean by baggage? Or linking to something to start research.
- UtopiaFans 3y ago
  [dead]
MichaelZuo 3y ago
This is a bit tangential to the author's point but it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason. Since it's so much easier to hide behind a new unique address, compared to IPv4, that any service such as Cloudflare would need to be extremely aggressive in blocking to meet their internal metrics and customer advertised minimum thresholds. So much so that it actually costs more to use IPv6 then sticking with IPv4. I imagine the scenario described by the author would become more and more common as time goes on as more of the world's internet users becomes harder to distinguish.
- johnklos 3y ago
  You're basically saying this behavior is acceptable and should be considered normal and should be expected to become the norm. If you think IPv6 is mostly pointless, I think you're unaware of the fact that a significant majority of phones already use IPv6 most of the time they're on cellular.
  - MichaelZuo 3y ago
    > You're basically saying this behavior is acceptable and should be considered normal and should be expected to become the norm. If you think IPv6 is mostly pointless, I think you're unaware of the fact that a significant majority of phones already use IPv6 most of the time they're on cellular. Can you point to where I suggested that? Or did you misread the comment?
    - warrenm 3y ago
      He quoted you: >it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason
      - MichaelZuo 3y ago
        > He quoted you: > >it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason Huh? There is no quote in that comment: https://news.ycombinator.com/item?id=37051011 https://news.ycombinator.com/item?id=37051011. Unless you are referring to a different comment?
thedaly 3y ago
I've been getting stuck in the “browser integrity check” loop a lot on firefox lately. Not an issue in chrome, not using a vpn, etc. I assume it is some combination of extensions and/or settings in firefox.
- IG_Semmelweiss 3y ago
  Its happened a few times here too I also have maxed out anti fingerprinting etc on FF, so it comes with the territory. I have to slowly enable JS on some sites to see if the loop will break, or i just navigate away. I use all browsers except chrome, but i only navigate the web with FF
alberth 3y ago
I'm surprised Apple PAT and Google WEI wasn't mentioned in the article. Especially since Apple has partnered with Cloudflare on PAT.
- joshstrange 3y ago
  They do mention it (Google's at least) in this section [0] [0] https://jrhawley.ca/2023/08/07/blocked-by-cloudflare#implications-for-the-web-in-general https://jrhawley.ca/2023/08/07/blocked-by-cloudflare#implica...
  - CharlesW 3y ago
    Also, the scope of PATs is vastly different than WEI. Think of PATs as a "probably a human" signal that mostly replaces the need for CAPTCHAs. https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas/ https://blog.cloudflare.com/how-to-enable-private-access-tok...
    - jsnell 3y ago
      Their scope is the same [0], both in terms of stated intent as well as what kind of things they could be used to attest. The only significant differences are that one is already deployed in prod while one isn't, one got a marketing blitz while the other didn't, and that they're done by different companies. There are no vast technical differences, only incredibly subtle ones. [0] https://www.snellman.net/blog/archive/2023-07-25-web-integrity-api-vs-private-access-tokens/ https://www.snellman.net/blog/archive/2023-07-25-web-integri...
jonatron 3y ago
Does anyone who knows about GDPR know if being blocked by a CDN comes under "Automated individual decision-making"?
- buro9 3y ago
  https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/rights-related-to-automated-decision-making-including-profiling/ https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re... That's interesting.
CharlesW 3y ago
> The next day, I tried accessing a web page internal to my company… […] I couldn’t get past a security check page because of issues in Cloudflare’s software. […] The silliness of it all is that I was on my work device the whole time, which was behind my workplace VPN. This seems more like an "IT department gone mad" problem than a Cloudflare problem. I'm surprised they'd rather switch to Chrome than submit a support ticket. Having used passkeys for a month+ now via macOS/iOS/1Password betas, I don't understand how they're related or the author's concerns. Couldn't you just replace "passkey" with "password" in all of their questions?
- tadfisher 3y ago
  Passkeys have optional attestation payloads, which is basically what WEI is doing. Google in particular doesn't recommend requiring attestation except in corporate-security scenarios, but the fear is that banking and media sites will require attestation anyway, which locks users into whatever attestation mechanisms supported by the server; so basically Google, Apple and Microsoft.
  - nullfield 3y ago
    You know I knew there was going to be something I really didn’t like about passkeys, and here it is
    - tadfisher 3y ago
      Yeah, unfortunately the point of passkeys is to replace multi-factor authentication. Usually you have a username+password as the primary factor, and a secret that's hard to copy and replay as a second factor (TOTP, non-resident WebAuthn credential/FIDO, SMS code). Passkeys replace the primary factor with a signed challenge, but the second factor is up to the authenticator (such as biometrics). WebAuthn relying parties verify that the authenticator is locking the primary factor behind the second factor, and they do that with attestation.
  - johncolanduoni 3y ago
    Does Microsoft have much control over Windows Hello's key attestation? It's not clear to me how they could pull that off, other than just relying on TPM attestations which are easy to obtain as long as you can buy a TPM that works with your motherboard.
zer8k 3y ago
Any time a large portion of internet traffic is controlled by a single source it brings problems like this with it. All cloudflare has to do is arbitrarily decide who and who can't use the internet and effectively their word becomes law. Like most things it starts with an innocent premise (e.g. "an easy way to stop bad actors") and ends up extended to any number of arbitrary things. Worse, the argument from privacy advocates rings hollow because defending privacy means you have to allow Bad People (TM). The average drooler using the internet cannot understand the nuance. Even in the most innocent of cases, a bad commit getting merged, can bring down the internet. It has happened before with Cloudflare. Companies like Cloudflare, Google, Meta, etc are the reason anti-trust law exists. Unfortunately, it appears there is no one with any power that is willing to use the laws for their purpose. The internet in 20 years will be nothing like we've seen before. That's not a good thing.
- NicoJuicy 3y ago
  Everyone forgets that websites become almost unusable because of crawlers and bots. Website owners specifically choose for cloudflare to protect against this, it's not forced upon them by cloudflare.
  - realusername 3y ago
    And who defines is an allowed crawler? Is Google the only allowed company to crawl the web? Isn't that the definition of monopoly? Could anybody still create a new search engine nowadays?
    - NicoJuicy 3y ago
      A valid crawler is following robots.txt
      - realusername 3y ago
        That's not going to be enough to pass Cloudflare.
AegirLeet 3y ago
I've had the exact same problem for a while. Here are some of the sites I've been unable to access (found by searching for "just a moment" in my browser history): - https://gitlab.com/users/sign_in https://gitlab.com/users/sign_in - https://steamdb.info/login/ https://steamdb.info/login/ - https://www.zabbix.com/forum/ https://www.zabbix.com/forum/ - https://casetext.com/ https://casetext.com/ - https://namemc.com/login https://namemc.com/login - https://spinroot.com/ https://spinroot.com/ - https://camelcamelcamel.com/ https://camelcamelcamel.com/ It's really annoying and Cloudflare is apparently doing nothing to fix it as this has been going on for months if not years. I guess Cloudflare just hates the open web and really wants to enforce Chrome/Chromium/Blink hegemony.
- megous 3y ago
  Yeah, gitlab also blocks me from logging in (via its cloudflare use). It did so even when we paid for it. We no longer do. (for other reasons, but anyway, good riddance)
- zer8k 3y ago
  I haven't had any problems on Waterfox. However, it is absurd to me I need javascript to simply visit a website anymore.
  - kmlx 3y ago
    that’s because websites have evolved into web apps.
    - adrianN 3y ago
      Most websites haven’t done that
  - michaelt 3y ago
    Sadly, the fact a given site works for you or me is no guarantee it works for someone else. These bot detection systems tend to use all manner of imprecise statistical heuristics and weird fingerprinting. Perhaps AegirLeet has a graphics card that a popular web scraper pretends to have. Maybe they're in a suspicious timezone. Maybe they've installed a font usually only found on a different operating system. Maybe I'm never blocked because I have an excellent IP reputation, due to regular visits to approved websites.
thedanbob 3y ago
Just yesterday I realized that I couldn't log into Paypal on Safari or Firefox, only a Chromium-based browser. We're getting deeper all the time into "this site is best viewed in Google Chrome".
- buro9 3y ago
  I've been experiencing the PayPal one for a while. Firefox on Windows, Linux or Android. Thankfully the app is still signed in, but I've used credit card for things that I have PayPal money just sitting there ready to spend as I can't get into PayPal on Firefox.
  - Animats 3y ago
    Time to switch from PayPal to FedNow. FedNow is run by banks and the Fed, which are regulated as to whom they can refuse to server.
    - buro9 3y ago
      Never seen an online shop accept that, nor do I know if it's open to UK citizens. But thanks
      - Animats 3y ago
        In the UK, you have Single European Payment Area payments. (Despite Brexit, the UK chose to stay within the SEPA zone.) The US didn't have a bank-level national service for consumer to consumer payments until FedNow. Just PayPal, Venmo, etc., which are second-tier services, which becomes an issue when they break.
        lmm 3y ago
        Still better than cutting off most of the world. PayPal have their problems but neither a US-only nor an EU-only system is an improvement.
    - 3y ago
derefr 3y ago
If you've ever tried to take apart Cloudflare's various session cookies, MITMed scripts sent for "high integrity" pages (or when in "super bot-fight" mode), etc., you'll have observed that it's basically running a web-worker to heuristically do browser-integrity checking. That is, Cloudflare is trying to run a series of tests that real browsers operated by users pass, but which headless browsers operated by bots will fail. These range from pretty simple things that check that the browser is actually a browser rather than a raw HTML parser (e.g. "draw an image on a <canvas>, export it to PNG, hash the PNG, compare to an expected result"); to things that check for low-effort headless-browsing techniques like the one you get by default using Puppeteer in a Lambda/Cloud Function (e.g. "do we have the weirder fonts you'd expect to exist on a consumer OS, but which these default batteries-included container images don't bother to bake in"); to things that work really hard to detect the "scent of humanity" through the browser (e.g. "before the user activated the integrity-check prompt, did we record a sequence of 'extraneous' mouse movements and key events that look like a human making individualized mistakes on their way to completing the form, and don't look like a recorded capture of such similar to other ones we've seen recently.") If you're getting caught in a verification loop, it's because you're using a browser or device or extension that obscures/disables enough of these heuristics that Cloudflare can't get proof positive that you're a person rather than a bot — and so, under whatever settings the site-owner has it set at, it will just keep trying to get that proof, rather than telling you you've failed and been blocked. (Why? Because telling a bot they've failed tells them that they should stop trying something that's not working and instead — in the words of Star Trek technobabble — "rotate their shield frequency" before trying again.)
- johnklos 3y ago
  How does that explain blocks that happen to less common browsers and/or less common platforms?
  - derefr 3y ago
    Heuristics are about optimizing between false positives and false negatives. Many headless-browser stealth techniques involve rotating between the signatures and reflected metrics of real — but niche and/or ancient — User-Agents. (For some reason, the developers of these stealth systems think that variety beats commonality. Maybe it makes sense if they're specifically trying to overcome Apache mod_security's signature-based UA blocking or something.) It turns out that when you actually see one of these UAs in your server logs, it's far more (99.99%) likely to be a stealthed bot that picked that UA out of a bag, than it is to be an actual niche/ancient UA. In the case of the niche UAs, this is a tragedy of the commons. In the case of the ancient UAs, though, there's no downside to blocking them entirely — because if the traffic is going through Cloudflare at all, then you're already requiring of the client a minimum version of TLS that the real old UAs can't even speak. So the only things actually saying they're that old device — but managing to get through an HTTP request at all — are stealthed bots.
w0ts0n 3y ago
Users in Egypt are unable to visit my Fitness website https://musclewiki.com https://musclewiki.com Cloudflare is a huge part of the internet. Often they won't respond and it appears that for whatever reason, their IP range is blocked in Egypt. We probably get 10 support emails per week. I contacted Cloudflare and they simply said there is nothing they can do.
- Temporary_31337 3y ago
  If you don’t want to stop using CloudFlare or need a temporary solution ask your users from Egypt to use VPN- many already do as they come across similar problems for other services
  - elashri 3y ago
    Egypt do block VPNs in much more aggressive way than Cloudflare. Also this is my first time to hear that cloudflare is blocked in Egypt. People will even complain that they cannot connect to their cooperate VPNs. Blocking cloudflare ip addresses means that half of the internet wouldn't be accessible from Egypt. its closw to blocking port 443 because some people use DNS over https. disclaimer: I'm Egyptian living in the US.
    - w0ts0n 3y ago
      Have someone in Egypt look at our site. Its been blocked for about 3 months. Apparently there is a pool of IP's that Cloudflare use for their CDN and some of them are blocked in Egypt. If you are unlucky enough to be one of the websites that is using that IP, it's blocked. Apparently they rotate them, but I haven't seen it yet, so chances are, when they do rotate them, more will be blocked.
- hiatus 3y ago
  I'm surprised to hear that. I actually used Cloudflare Tunnel to connect to a corporate intranet about 8 months ago while in Egypt, not sure if things have changed though.
- warrenm 3y ago
delfinom 3y ago
Yea I noticed the same thing for awhile. Cloudflare actively blocks non-Chrome browsers.
- luuurker 3y ago
  I use Firefox (stable and dev) and Waterfox, and Cloudflare doesn't block me. My settings are close to the defaults though, I don't enable things like privacy.resistFingerprinting. For a while I did notice that when using certain IP blocks, they would show me more captchas when using Firefox than when using Chrome, but I haven't had that problem in a while.
- tracker1 3y ago
  https://addons.mozilla.org/en-US/firefox/addon/privacy-pass/ https://addons.mozilla.org/en-US/firefox/addon/privacy-pass/
buzer 3y ago
It's also annoying how that check page ends up breaking page reload in Firefox. When Cloudflare redirects you back to the page it will happen via POST. This initial POST gets captured by Cloudflare, but if you reload the page that POST will go to page itself and there's pretty good chance it doesn't know what to do with that and just shows error. The only fix is to navigate back to page somehow, either by going to address bar and pressing enter (to navigate there again instead of reloading) or finding some link that points you back to the page. I wouldn't be surprised if those POSTs will end up banning you from some website since they "know" you shouldn't POSTing to that page so clearly you are evil bot trying to hack them.
tamimio 3y ago
I was going to assume that the corp VPN is the reason as maybe someone is abusing that connection for something else and it’s getting flagged, but the fact that the site worked using chrome says otherwise. Will using chromium for such cases work while having Firefox for the rest of sites? And what’s cloudflare alternative that provides similar services for free including traffic analysis?
ChrisArchitect 3y ago
Is there something more going on here like you're using some kind of blocker and it's stopping the captcha/security 'widget' from loading?
j16sdiz 3y ago
> Worse yet, I know that Cloudflare knows I have those certificates. Why? Because it asked for them! It doesn’t make sense for Cloudflare to request any client certificates. I think there are real bugs somewhere.
- ArchOversight 3y ago
  Cloudflare allows you to enable mTLS for websites: https://developers.cloudflare.com/ssl/client-certificates/enable-mtls https://developers.cloudflare.com/ssl/client-certificates/en... https://developers.cloudflare.com/cloudflare-one/identity/devices/access-integrations/mutual-tls-authentication/ https://developers.cloudflare.com/cloudflare-one/identity/de... This would then require Cloudflare to request a client certificate. This is great for securing websites using corporate identity that is derived from AD certs for example to make sure the device being used has a valid cert on it. Alongside MDM for example forcing the certificate to have a short lifespan (my $CORP uses 7 days) you can validate that the device has the correct security posture to access the resources. If for example I let my device not update the version of macOS often enough my cert expires and I can't access internal resources until I update my OS and MDM software checks that and provisions me a new device certificate.
- Operyl 3y ago
  It's requesting client certs for their internal intranet stuff hosted behind cloudflare.
miyuru 3y ago
I cannot access flyertalk.com, which hosts lot of useful airline content from any IP from my country. I tried reaching out via email as mentioned in the error page and admin does even have a valid email posted anywhere. I know cloudflare is not to blame here, but they provide way easy access to blocking to bad admins.
Dwedit 3y ago
The big problem I have with Cloudflare's integrity check is that all the spam domains use a fake version which mimics it, and tries to trick you into completing a captcha.
superkuh 3y ago
This has been my experience for a handful of years but of course it's getting worse. In the past I'd just be getting blocked from access commercial websites or applications and things I didn't really need. But in the last few years many scientific publishers have put all their content behind cloudflare walls. Pretty much my only hope of being able to read a paper these days is that it came out long enough ago to be on sci-hub or they published the pre-print on arxiv/bioarxiv/etc. Once arxiv goes behind a cloudflare I don't know what I'll do.
tracker1 3y ago
For good or bad, this is why I have the Privacy Pass extension installed.
gsich 3y ago
"Checking if the site connection is secure" what a blatant lie. If you can read this you already have a TLS connection.
adammartinetti 3y ago
Hi there, I'm the PM for Cloudflare's challenge platform. I'd love to look into what the cause of the problem is, so you don't see these difficulties. > Cloudflare detected the high frequency of requests and denials (but not their faulty loop that caused this pattern of requests, of course), and tagged my browser as suspicious. I can tell you at least that we don't penalize users for this looping behavior, so this wouldn't cause us to see your browser as suspicious. I hope we can dig into this more and uncover the cause of the problem. Personally, I'm a big Firefox user, and this isn't behavior I see. If there were a widespread Firefox wide issue, automated alerts would trigger and we'd consider this a critical incident. You can drop me an email at amartinetti at cloudflare if you're interested in troubleshooting.
- bradly 3y ago
  Anecdotaly... I use Firefox and have noticed the Cloudflare interception pages verifying I'm human appearing more often recently. Usually it is all automatic and isn't a big deal, but I have noticed a increase in how often I see these the past week.
  - deleted 3y ago
    [deleted]
  - stjohnswarts 3y ago
    I use a VPN 99% of the time and I definitely see more cloudflare on my systems (as opposed to when I'm bored and surfing in the library) and see the "checking" screen. It does seem to be a lot better than last year though when it seemed I was getting a captcha every time I turned around
    - JakeAl 3y ago
      I have the same issue and often just reconnecting using a different node/city resolves it Annoying as hell, but sometimes the problem is the nodes in the US show as being in EU for some reason.
  - lovegrenoble 3y ago
bradley13 3y ago
Anecdote: For my programming classes, one example I use is a simple browser. It doesn't do CSS or Javacript, so display is primitive, but it works. On some sites. Many sites, especially the big ones, see that it's an unknown browser, and refuse to send content. Probably they think it's a bot. But even if it were, what's wrong with bots, as long as they're well-behaved? What kind of closed web have we let the megacorps build?
- unmole 3y ago
  Playing the devil's advocate: Why shouldn't a server get to decide which clients it wants to talk to?
  - userbinator 3y ago
    Why shouldn't a store get to decide the $protected_class of which customers it will do business with?
    - unmole 3y ago
      User Agent is not a protected class. Neither is Intentionally Obtuse for that matter.
  - 3LazTjBv-f 3y ago
    In principle? Of course. I mean I remember blocking Yandex bots from hammering some e-commerce site on shoestring budget. But each person developing a web scrapping bot realizes at most after a week that being honest with User-agent has negative impact on how well it works, and changing it to existing browser takes literally seconds.
  - doctor_radium 3y ago
    Long term, I can imagine this becoming a corollary to Net Neutrality. Whenever that finally becomes a thing, the next step could be a law/rule that "public web sites need to be accessible by the public". Not that developers need to test their work in dozens of different browsers, but that they can't actively choose their customers.
krono 3y ago
The amount of times Cloudflare is making me sit through their 15 to 30 second "checking your connection" page is insane. For people going through life with ADHD such as myself, the impact of all these delays and disruptions throughout the day can be severe. Despite being properly medicated this measure is absolutely debilitating and makes for a dreadful and very taxing online experience.
- jeroenhd 3y ago
  Cloudflare's Privacy Pass may help here: https://privacypass.github.io/ https://privacypass.github.io/ It should significantly reduce the amount of CAPTCHAs you see in a way that's not terrible for privacy. For Safari, you can enable Private Access Tokens: https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas/ https://blog.cloudflare.com/how-to-enable-private-access-tok... Both of these mechanisms are similar to Google's web DRM proposal in that they rely on external issuers to generate tokens, but unlike Google's attempt they don't guarantee that ad blockers are disabled on pages that try to use tokens.
  - tomxor 3y ago
    Wow, that doesn't sound like a terrible idea! Which is honestly surprising in this area where it feels like privacy, anonymity and human verification are incompatible with each other. I am trying to minimise my time wasted by websites, which is hard to balance with privacy, one other one is the repetitive consent forms (if you don't retain cookies, it's a never ending process). I think consent forms and human verification are the 2 biggest human time wasters.
    - jeroenhd 3y ago
      > Which is honestly surprising in this area where it feels like privacy, anonymity and human verification are incompatible with each other. The thing is, it still allows for some correlation between attestation provider and the websites themselves, potentially exposing part of your browsing history to these companies based on how many tokens you use and what websites consume them. That doesn't matter much for Cloudflare's implementation (now Cloudflare knows when you visit Cloudflare, oh no!) but with Apple's attestation provider the risks increase. The smaller the attestation provider gets or the fewer parties trust that particular attestation provider, the higher the risk becomes. It's better for your privacy than the current norm (de-anonimisation through fingerprinting while you fill out a CAPTCHA) but it's still not great. It also allows for attestation providers (and their algorithms) to arbitrarily deny you access to the web if other websites decide to start using them. Privacy in exchange for power, I'm not so sure about that. I imagine for someone suffering from ADHD the small risk that Cloudflare decides to screw you in particular is worth the massive improvement in browsing experience, but everyone will have to determine the pros and cons for themselves.
dcow 3y ago
So many privacy nuts use Chrome and don't realize this: > What about Google Chrome? > I tried all of the above in Firefox. So I naturally tried to access the same page in Google Chrome to see if I’d still be blocked. Thankfully, I wasn’t. > But of course I wasn’t because Chrome doesn’t have the same privacy- and security-enhancing designs that Firefox does. Chrome will happily collect as much private information about me and my browsing history and share them with select parties, as needed. It also doesn’t resist fingerprinting or let me modify settings to the same degree that Firefox does because Chrome relies on those fingerprinting technologies to ensure that I am targeted by ads it deems necessary for me to see. > Being blocked on Firefox and not blocked on Chrome also tells me that Cloudflare is blocking me based on the fingerprint (or lackthereof) of my browser. Everything about my connection is identical between the two requests, aside from the browser being used. It’s the same security certificates, same corporate VPN, same machine, even the same timeframe when I try to access the site. If you care about anything these days, don't use Chrome.
- afavour 3y ago
  I’m no Google fanboy but I wasn’t satisfied with this: > Chrome will happily collect as much private information about me and my browsing history and share them with select parties, as needed What information does Chrome provide in this scenario that Firefox doesn’t? It feels like backward logic: it worked in Chrome therefore it must be because Chrome gave extra info. In reality it could be a whole bunch of things, something as mundane as Firefox being a rarer user agent so subject to more filtering. It strikes me that all of this is an inexact science. I've run into rate limit messages with sites before now that go away when I switch browsers, no matter what the browser is. I assume it's because, with the limited information given, the DDOS protection software assumes that same IP + different UA = different computer. I have no clue but I wasn’t persuaded that this specific scenario works with Chrome because it was giving away more information. At a bare minimum at least try a third browser!
  - deadbunny 3y ago
    You're conflating a downside of using Chrome and the reason they think Cloudflare blocked them.
ricardo81 3y ago
Boils down to gatekeepers doesn't it. Unfortunately there's also bad actors on the web (and the definition of bad varies). I understand reasons to try centralise the removal of that so called bad, but obviously a central group deciding on the 'bad' just isn't democratic. Ironically when chatgpt mentioned their UA on a web page the other day, users were presented with an anti-bot challenge.
Animats 3y ago
Suing Cloudflare for interference with contract[1] might be an option. Cloudflare is not protected against lawsuits by some EULA, because the outside user has no contract with them. They're a third party in the middle. Talk to a lawyer. Most contract law lawsuits are settled out of court. The great advantage of suing someone is that you get past the low-level customer support people and talk to someone who's authorized to settle. [1] https://www.lodhs.com/blog/interference-with-contractual-or-advantageous-relationships/ https://www.lodhs.com/blog/interference-with-contractual-or-...
- HumanOstrich 3y ago
  Can you provide examples of people suing Cloudflare to get around being blocked from accessing certain websites?
rejectfinite 3y ago
Using something like Edge (for work), Vivaldi or Brave would be better than Chrome and probably let you in instead of Chrome.
jeroenhd 3y ago
Some comments I have on this post: > Worse yet, I know that Cloudflare knows I have those certificates. Why? Because it asked for them! Not really. Cloudflare notices your browser has TLS authentication available and asks you for it. That's really annoying, but part of the protocol spec. Your browser won't send this information unless you pick a certificate and hit OK. Disable your ad blocker and you'll find that many trackers will also ask you to identify yourself this way. It's really annoying, browsers need to design better UX for this type of authentication. > · MAC address of my machine that I have previously used to access this site How does it gather your MAC address? Did you disable IPv6 Privacy Extensions? Unless the website is sitting behind the same switch as your computer or you run some kind of native application that sends the MAC address, websites can't read the MAC of your network interface. Enable the MAC randomisation that's present (sometimes even turned on by default!) in every modern OS if you consider the local switch or WiFi network to be a privacy risk. > Will I be able to create and sync these passkeys myself? Yes, assuming they follow the standard > Can only certain types of software use passkeys? If so, who decides what software meets this standard? I don't really understand the question. Any software supporting passkeys will be able to prompt you for generating or using a passkey. > Will I only be able to generate passkeys on a device with specific hardware/software requirements like a TPM, DeviceCheck, or Integrity API? According to the spec, keys can be stored in software no trouble. Websites and apps can ask for securely generated keys, but I don't think those are all that common. Hardware can also be faked relatively easily in most circumstances. > Can I, at any time, export my passkeys from one service provider and switch to another provider? Ask your service provider for export options. Most likely, you can't just dump the keys and import them elsewhere (that would defeat the point). > If a passkey is invovled in a suspicious event, will that suspicious mark propogate to any other device that uses that same passkey? Do devices that contain suspicious passkeys also get marked as suspicious? If so, would that impact the ability of that device to access other independent websites? That depends on the software using the key for authentication. Maybe?
- paradox460 3y ago
  I've seen the misconception a lot; people seem to think that random websites can grab your MAC like they can get your IP address.
rubatuga 3y ago
Actually cloudflare doesn't have access to your MAC address so it's a bit more difficult to attest that you are a legitimate user.
deleted 3y ago
[deleted]
xmichael909 3y ago
Cloudflare sucks, plain and simple, no idea why anyone uses it. So many better alternatives.
- aeyes 3y ago
  For example? Especially when taking price into account.
koito17 3y ago
Increasingly more sites are getting stuck in a Cloudflare verification loop on my end. I use Firefox on the beta channel, and I do have a few privacy extensions and a heavily modified user.js. If you want to give in to the browser fingerprinting, I have found that enabling WebGL, enabling performance timing (wow), setting network.http.referer.XOriginTrimmingPolicy to 0, among other tweaks, helps me break out of the verification loop. In other words, if Cloudflare can't reliably fingerprint your browser, you are treated as a "bot" and denied access to a huge chunk of the web. Well, in that case, I would rather be a bot than a human. Being a human seems to be increasingly annoying nowadays :)
parasti 3y ago
Insanely enough, Cloudflare sometimes puts these pages in front of API endpoints, as if that JSON were for human eyes only.
adamgamble 3y ago
One thing that sometimes gets lost is site owners that use cloudflare have sort of global options for how paranoid they want to be, then they can make specific WAF rules that can be as granular and aggressive as they want. So at least in some cases, cloudflare gets blamed for website owners setting really aggressive rules. The effect on the end user usually looks exactly the same. Case in point, I set a waf rule that blocked all non verified bot traffic from several big datacenters (Google cloud, OVH, digital ocean, etc). That turned out to be a mistake because a lot of corporations were routing their traffic through those ASNs for some reason. Now they’re blocked. They could have gotten pissed out cloudflare, the error page looks the same, but it was really misconfiguring it.
skybrian 3y ago
Fingerprinting is probably load-bearing for captchas and other anti-fraud stuff that many Internet services and businesses depend on: https://xkcd.com/2347/ https://xkcd.com/2347/ It should be replaced with something better. Unfortunately all attempts to do something better get attacked by people who don’t realize that you can’t just get rid of it, or important things will break.
nfriedly 3y ago
> Anyone who uses a de-Googled Android phone has to go to great lengths to ensure hardware attestation is working correctly [...] or else they can’t using banking apps. I have a relatively Google'd Android running lineageOS. It passes SafteyNet on a fresh install, but even that isn't good enough for one of my banking apps (or netflix) - they both also perform a CTS Profile (Compatibility Test Suite) check and block me from using the app if they don't like what they see. I ultimately had to root the phone to be able to use my bank's app. Rooting allowed me to use a fake CTS Profile, and then because it was rooted, SafteyNet started failing and I had to install a bypass to work around that. Now everything works great, except OS updates un-root the phone and then "secure" apps stop working again. (Oh, and if you mention that you're rooted, the LineageOS folks will refuse to provide any support, even for unrelated issues. Making you choose between friendly help and a usable phone is probably the only thing I don't like about LineageOS and, to my view, the biggest break from it's CyanogenMod roots.)
amatecha 3y ago
On one of my machines I run OpenBSD with Firefox with "Strict" privacy settings and "privacy.resistFingerprinting" enabled. There are so many websites I can't access, I get a straight-up 403 Forbidden page because CloudFlare has decided I am not trustworthy. I mean like pretty big companies like DigiKey, Home Depot, Canadian Tire, etc. I simply cannot use their websites, or I can load the initial page but then the API calls that provide the functionality all fail with a 403. DigiKey did something to unblock me and I can use their site again, but I know it's just a matter of time before it happens again. It's also a frequent problem on smaller sites that are simply using CloudFlare , and I never know when I'm going to be blocked from a site arbitrarily. It's especially egregious when it's a plain old text-based site like hamuniverse.com , or a small independent vendor like digirig.net ... This is one of the things that makes it so clear to me that the web is diverging into two, one that is the "clean walled-garden capitalism web" and the continuation of the original web that was open, freely-accessible and built around sharing and knowledge.
kelnos 3y ago
Prediction: if Google manages to ram Web Environment Integrity down our throats, CloudFlare will implement it as a part of these checks.
lopkeny12ko 3y ago
Anyone else find it odd that the author's company-internal work intranet, which requires a VPN to access, is deployed behind a Cloudflare CDN? Why would anyone do this?
deleted 3y ago
[deleted]
1vuio0pswjnm7 3y ago
No problem for me accessing Gitlab without using a web browser. Moreover one can use Internet Archive, Archive.today, Google's cache, etc. to avoid SNI. The author did not specify which project he was trying to access so I picked a random one to test from /explore/projects/topics/bioinformatics. No problem accessing it without a web browser. TLS1.3. No SNI. https://one-touch-pipeline.gitlab.io/otp/ https://one-touch-pipeline.gitlab.io/otp/
samcat116 3y ago
The fact that this person thinks Cloudflare has their MAC address leads me to believe they shouldn't be speculating on the "implications for the web"
datavirtue 3y ago
Please rename to "Blocked by Firefox"
Havoc 3y ago
Yeah also noticed I could t get past certain sites recently
simple-thoughts 3y ago
I personally experience this loop all the time on different sites. I’ve completely given up - if a site loops I don’t use it and try again a few weeks later. If it’s something extremely urgent I use my mobile device which for some unknown reason never loops.
deleted 3y ago
[deleted]
nhanpq 3y ago
I feel like this these days: the right to decide if I am free to choose to use or access a service or website is not based on whether I claim to be human (in captchas tests), but based on the data people collect about me - and decide on them - something that I don't know behind invisible doors. I thought privacy was on the rise after the data leaks and irresponsibility of the big tech companies, and the public's involvement in the issue of individual privacy, but it seems like everything is still a step backwards.
issung 3y ago
I use cloudflare for hosting my sites, can I and how would I disable this functionality?
foxylad 3y ago
Off topic - but how can Cloudflare block access to an internal website when accessed by via VPN? And if CF has some kind of request verification API that the internal server is using, why would you use it for an internal resource?
- doctor_radium 3y ago
  [dead]
phreack 3y ago
This happened to me just a few days ago. I tried to open a link in an app, which then tried to open it with an in-app WebView. Thus, the Cloudflare captcha loop of death. I could even see a "human verification failed" string appearing after clicking on an "I am human" checkbox. Alongside the annoyance of not being able to browse, this kind of language is awful! Literally being told to my face I am not a human.
Tozen 3y ago
The problem with Cloudflare is they purposely attempt to break user privacy by dangling websites as carrots. It's deception that they are attempting to determine if the person is human or not, because often they won't even show you the CAPTCHA. Even if you do get to the CAPTCHA, sucessfully doing it usually won't give you access to the website either. So, what is the point? They want people to disable any privacy protections or push usage of browsers that have no to less privacy protections, in order to access the website they are blocking. This has nothing to do with if a user is an actual threat or bot, but is more a strategy to shape what browsers are used and destroy user privacy. Cloudflare is also very aware of the numerous and constant complaints about what they are doing, coming from users and for years. They are ignored, because they have something else in mind.
1vuio0pswjnm7 3y ago
The End of the Road for Cloudflare CAPTCHAs https://blog.cloudflare.com/end-cloudflare-captcha/ https://blog.cloudflare.com/end-cloudflare-captcha/ Discriminate against all but "major browsers". Why. https://developers.cloudflare.com/fundamentals/get-started/concepts/cloudflare-challenges/ https://developers.cloudflare.com/fundamentals/get-started/c...
fartcanister 3y ago
The same for Safari. If your website is available in only one browser, this means that you need to change the content delivery provider.
monsieurUK 3y ago
I use ungoogled chromium on my openBSD machine, and I think it's better for privacy than firefox :-).
Jaskolka 3y ago
Fuck Cloudflare
rcMgD2BwE72F 3y ago
As a Firefox mobile user, I've never been able to go past that page for ~2 years. And so I've stopped visiting websites that use that system (several per days). There's no way to report that to Cloudflare so f*ck'em.

632 points by jrhawley 3y ago

[flagged]

> I temporarily disabled extensions. I opened a private browsing window. FYI When using Chrome, incognito window carries a lot of baggage. For issues like this use Guest profile as it doesn't include extensions, caches, storage, etc. Optionally do a Google search first to seed it with cookies.

jsnell 3y ago
Doing a Google search will not populate any cookies Cloudflare could access.
- kozhevnikov 3y ago
  Of course not, but it is less likely to trigger manual recaptcha on the site you're trying to visit in Guest mode.
  - Operyl 3y ago
    Cloudflare does not use Google's reCAPTCHA anymore, and hasn't for some time.
fireflash38 3y ago
Mind expanding on what you mean by baggage? Or linking to something to start research.
UtopiaFans 3y ago
[dead]

MichaelZuo 3y ago

This is a bit tangential to the author's point but it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason. Since it's so much easier to hide behind a new unique address, compared to IPv4, that any service such as Cloudflare would need to be extremely aggressive in blocking to meet their internal metrics and customer advertised minimum thresholds. So much so that it actually costs more to use IPv6 then sticking with IPv4. I imagine the scenario described by the author would become more and more common as time goes on as more of the world's internet users becomes harder to distinguish.

johnklos 3y ago
You're basically saying this behavior is acceptable and should be considered normal and should be expected to become the norm. If you think IPv6 is mostly pointless, I think you're unaware of the fact that a significant majority of phones already use IPv6 most of the time they're on cellular.
- MichaelZuo 3y ago
  > You're basically saying this behavior is acceptable and should be considered normal and should be expected to become the norm. If you think IPv6 is mostly pointless, I think you're unaware of the fact that a significant majority of phones already use IPv6 most of the time they're on cellular. Can you point to where I suggested that? Or did you misread the comment?
  - warrenm 3y ago
    He quoted you: >it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason
    - MichaelZuo 3y ago
      > He quoted you: > >it does seem to indicate that IPv6 is mostly pointless for human users for exactly this reason Huh? There is no quote in that comment: https://news.ycombinator.com/item?id=37051011 https://news.ycombinator.com/item?id=37051011. Unless you are referring to a different comment?

thedaly 3y ago

I've been getting stuck in the “browser integrity check” loop a lot on firefox lately. Not an issue in chrome, not using a vpn, etc. I assume it is some combination of extensions and/or settings in firefox.

IG_Semmelweiss 3y ago
Its happened a few times here too I also have maxed out anti fingerprinting etc on FF, so it comes with the territory. I have to slowly enable JS on some sites to see if the loop will break, or i just navigate away. I use all browsers except chrome, but i only navigate the web with FF

alberth 3y ago

I'm surprised Apple PAT and Google WEI wasn't mentioned in the article. Especially since Apple has partnered with Cloudflare on PAT.

joshstrange 3y ago
They do mention it (Google's at least) in this section [0] [0] https://jrhawley.ca/2023/08/07/blocked-by-cloudflare#implications-for-the-web-in-general https://jrhawley.ca/2023/08/07/blocked-by-cloudflare#implica...
- CharlesW 3y ago
  Also, the scope of PATs is vastly different than WEI. Think of PATs as a "probably a human" signal that mostly replaces the need for CAPTCHAs. https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas/ https://blog.cloudflare.com/how-to-enable-private-access-tok...
  - jsnell 3y ago
    Their scope is the same [0], both in terms of stated intent as well as what kind of things they could be used to attest. The only significant differences are that one is already deployed in prod while one isn't, one got a marketing blitz while the other didn't, and that they're done by different companies. There are no vast technical differences, only incredibly subtle ones. [0] https://www.snellman.net/blog/archive/2023-07-25-web-integrity-api-vs-private-access-tokens/ https://www.snellman.net/blog/archive/2023-07-25-web-integri...

jonatron 3y ago

Does anyone who knows about GDPR know if being blocked by a CDN comes under "Automated individual decision-making"?

buro9 3y ago
https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/rights-related-to-automated-decision-making-including-profiling/ https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re... That's interesting.

CharlesW 3y ago

> The next day, I tried accessing a web page internal to my company… […] I couldn’t get past a security check page because of issues in Cloudflare’s software. […] The silliness of it all is that I was on my work device the whole time, which was behind my workplace VPN. This seems more like an "IT department gone mad" problem than a Cloudflare problem. I'm surprised they'd rather switch to Chrome than submit a support ticket. Having used passkeys for a month+ now via macOS/iOS/1Password betas, I don't understand how they're related or the author's concerns. Couldn't you just replace "passkey" with "password" in all of their questions?

tadfisher 3y ago
Passkeys have optional attestation payloads, which is basically what WEI is doing. Google in particular doesn't recommend requiring attestation except in corporate-security scenarios, but the fear is that banking and media sites will require attestation anyway, which locks users into whatever attestation mechanisms supported by the server; so basically Google, Apple and Microsoft.
- nullfield 3y ago
  You know I knew there was going to be something I really didn’t like about passkeys, and here it is
  - tadfisher 3y ago
    Yeah, unfortunately the point of passkeys is to replace multi-factor authentication. Usually you have a username+password as the primary factor, and a secret that's hard to copy and replay as a second factor (TOTP, non-resident WebAuthn credential/FIDO, SMS code). Passkeys replace the primary factor with a signed challenge, but the second factor is up to the authenticator (such as biometrics). WebAuthn relying parties verify that the authenticator is locking the primary factor behind the second factor, and they do that with attestation.
- johncolanduoni 3y ago
  Does Microsoft have much control over Windows Hello's key attestation? It's not clear to me how they could pull that off, other than just relying on TPM attestations which are easy to obtain as long as you can buy a TPM that works with your motherboard.

zer8k 3y ago

Any time a large portion of internet traffic is controlled by a single source it brings problems like this with it. All cloudflare has to do is arbitrarily decide who and who can't use the internet and effectively their word becomes law. Like most things it starts with an innocent premise (e.g. "an easy way to stop bad actors") and ends up extended to any number of arbitrary things. Worse, the argument from privacy advocates rings hollow because defending privacy means you have to allow Bad People (TM). The average drooler using the internet cannot understand the nuance. Even in the most innocent of cases, a bad commit getting merged, can bring down the internet. It has happened before with Cloudflare. Companies like Cloudflare, Google, Meta, etc are the reason anti-trust law exists. Unfortunately, it appears there is no one with any power that is willing to use the laws for their purpose. The internet in 20 years will be nothing like we've seen before. That's not a good thing.

NicoJuicy 3y ago
Everyone forgets that websites become almost unusable because of crawlers and bots. Website owners specifically choose for cloudflare to protect against this, it's not forced upon them by cloudflare.
- realusername 3y ago
  And who defines is an allowed crawler? Is Google the only allowed company to crawl the web? Isn't that the definition of monopoly? Could anybody still create a new search engine nowadays?
  - NicoJuicy 3y ago
    A valid crawler is following robots.txt
    - realusername 3y ago
      That's not going to be enough to pass Cloudflare.

AegirLeet 3y ago

I've had the exact same problem for a while. Here are some of the sites I've been unable to access (found by searching for "just a moment" in my browser history): - https://gitlab.com/users/sign_in https://gitlab.com/users/sign_in - https://steamdb.info/login/ https://steamdb.info/login/ - https://www.zabbix.com/forum/ https://www.zabbix.com/forum/ - https://casetext.com/ https://casetext.com/ - https://namemc.com/login https://namemc.com/login - https://spinroot.com/ https://spinroot.com/ - https://camelcamelcamel.com/ https://camelcamelcamel.com/ It's really annoying and Cloudflare is apparently doing nothing to fix it as this has been going on for months if not years. I guess Cloudflare just hates the open web and really wants to enforce Chrome/Chromium/Blink hegemony.

megous 3y ago
Yeah, gitlab also blocks me from logging in (via its cloudflare use). It did so even when we paid for it. We no longer do. (for other reasons, but anyway, good riddance)
zer8k 3y ago
I haven't had any problems on Waterfox. However, it is absurd to me I need javascript to simply visit a website anymore.
- kmlx 3y ago
  that’s because websites have evolved into web apps.
  - adrianN 3y ago
    Most websites haven’t done that
- michaelt 3y ago
  Sadly, the fact a given site works for you or me is no guarantee it works for someone else. These bot detection systems tend to use all manner of imprecise statistical heuristics and weird fingerprinting. Perhaps AegirLeet has a graphics card that a popular web scraper pretends to have. Maybe they're in a suspicious timezone. Maybe they've installed a font usually only found on a different operating system. Maybe I'm never blocked because I have an excellent IP reputation, due to regular visits to approved websites.

thedanbob 3y ago

Just yesterday I realized that I couldn't log into Paypal on Safari or Firefox, only a Chromium-based browser. We're getting deeper all the time into "this site is best viewed in Google Chrome".

buro9 3y ago
I've been experiencing the PayPal one for a while. Firefox on Windows, Linux or Android. Thankfully the app is still signed in, but I've used credit card for things that I have PayPal money just sitting there ready to spend as I can't get into PayPal on Firefox.
- Animats 3y ago
  Time to switch from PayPal to FedNow. FedNow is run by banks and the Fed, which are regulated as to whom they can refuse to server.
  - buro9 3y ago
    Never seen an online shop accept that, nor do I know if it's open to UK citizens. But thanks
    - Animats 3y ago
      In the UK, you have Single European Payment Area payments. (Despite Brexit, the UK chose to stay within the SEPA zone.) The US didn't have a bank-level national service for consumer to consumer payments until FedNow. Just PayPal, Venmo, etc., which are second-tier services, which becomes an issue when they break.
      - lmm 3y ago
        Still better than cutting off most of the world. PayPal have their problems but neither a US-only nor an EU-only system is an improvement.
  - 3y ago

derefr 3y ago

If you've ever tried to take apart Cloudflare's various session cookies, MITMed scripts sent for "high integrity" pages (or when in "super bot-fight" mode), etc., you'll have observed that it's basically running a web-worker to heuristically do browser-integrity checking. That is, Cloudflare is trying to run a series of tests that real browsers operated by users pass, but which headless browsers operated by bots will fail. These range from pretty simple things that check that the browser is actually a browser rather than a raw HTML parser (e.g. "draw an image on a <canvas>, export it to PNG, hash the PNG, compare to an expected result"); to things that check for low-effort headless-browsing techniques like the one you get by default using Puppeteer in a Lambda/Cloud Function (e.g. "do we have the weirder fonts you'd expect to exist on a consumer OS, but which these default batteries-included container images don't bother to bake in"); to things that work really hard to detect the "scent of humanity" through the browser (e.g. "before the user activated the integrity-check prompt, did we record a sequence of 'extraneous' mouse movements and key events that look like a human making individualized mistakes on their way to completing the form, and don't look like a recorded capture of such similar to other ones we've seen recently.") If you're getting caught in a verification loop, it's because you're using a browser or device or extension that obscures/disables enough of these heuristics that Cloudflare can't get proof positive that you're a person rather than a bot — and so, under whatever settings the site-owner has it set at, it will just keep trying to get that proof, rather than telling you you've failed and been blocked. (Why? Because telling a bot they've failed tells them that they should stop trying something that's not working and instead — in the words of Star Trek technobabble — "rotate their shield frequency" before trying again.)

johnklos 3y ago
How does that explain blocks that happen to less common browsers and/or less common platforms?
- derefr 3y ago
  Heuristics are about optimizing between false positives and false negatives. Many headless-browser stealth techniques involve rotating between the signatures and reflected metrics of real — but niche and/or ancient — User-Agents. (For some reason, the developers of these stealth systems think that variety beats commonality. Maybe it makes sense if they're specifically trying to overcome Apache mod_security's signature-based UA blocking or something.) It turns out that when you actually see one of these UAs in your server logs, it's far more (99.99%) likely to be a stealthed bot that picked that UA out of a bag, than it is to be an actual niche/ancient UA. In the case of the niche UAs, this is a tragedy of the commons. In the case of the ancient UAs, though, there's no downside to blocking them entirely — because if the traffic is going through Cloudflare at all, then you're already requiring of the client a minimum version of TLS that the real old UAs can't even speak. So the only things actually saying they're that old device — but managing to get through an HTTP request at all — are stealthed bots.

w0ts0n 3y ago

Users in Egypt are unable to visit my Fitness website https://musclewiki.com https://musclewiki.com Cloudflare is a huge part of the internet. Often they won't respond and it appears that for whatever reason, their IP range is blocked in Egypt. We probably get 10 support emails per week. I contacted Cloudflare and they simply said there is nothing they can do.

Temporary_31337 3y ago
If you don’t want to stop using CloudFlare or need a temporary solution ask your users from Egypt to use VPN- many already do as they come across similar problems for other services
- elashri 3y ago
  Egypt do block VPNs in much more aggressive way than Cloudflare. Also this is my first time to hear that cloudflare is blocked in Egypt. People will even complain that they cannot connect to their cooperate VPNs. Blocking cloudflare ip addresses means that half of the internet wouldn't be accessible from Egypt. its closw to blocking port 443 because some people use DNS over https. disclaimer: I'm Egyptian living in the US.
  - w0ts0n 3y ago
    Have someone in Egypt look at our site. Its been blocked for about 3 months. Apparently there is a pool of IP's that Cloudflare use for their CDN and some of them are blocked in Egypt. If you are unlucky enough to be one of the websites that is using that IP, it's blocked. Apparently they rotate them, but I haven't seen it yet, so chances are, when they do rotate them, more will be blocked.
hiatus 3y ago
I'm surprised to hear that. I actually used Cloudflare Tunnel to connect to a corporate intranet about 8 months ago while in Egypt, not sure if things have changed though.
warrenm 3y ago

delfinom 3y ago

Yea I noticed the same thing for awhile. Cloudflare actively blocks non-Chrome browsers.

luuurker 3y ago
I use Firefox (stable and dev) and Waterfox, and Cloudflare doesn't block me. My settings are close to the defaults though, I don't enable things like privacy.resistFingerprinting. For a while I did notice that when using certain IP blocks, they would show me more captchas when using Firefox than when using Chrome, but I haven't had that problem in a while.
tracker1 3y ago
https://addons.mozilla.org/en-US/firefox/addon/privacy-pass/ https://addons.mozilla.org/en-US/firefox/addon/privacy-pass/

buzer 3y ago

It's also annoying how that check page ends up breaking page reload in Firefox. When Cloudflare redirects you back to the page it will happen via POST. This initial POST gets captured by Cloudflare, but if you reload the page that POST will go to page itself and there's pretty good chance it doesn't know what to do with that and just shows error. The only fix is to navigate back to page somehow, either by going to address bar and pressing enter (to navigate there again instead of reloading) or finding some link that points you back to the page. I wouldn't be surprised if those POSTs will end up banning you from some website since they "know" you shouldn't POSTing to that page so clearly you are evil bot trying to hack them.

tamimio 3y ago

I was going to assume that the corp VPN is the reason as maybe someone is abusing that connection for something else and it’s getting flagged, but the fact that the site worked using chrome says otherwise. Will using chromium for such cases work while having Firefox for the rest of sites? And what’s cloudflare alternative that provides similar services for free including traffic analysis?

ChrisArchitect 3y ago

Is there something more going on here like you're using some kind of blocker and it's stopping the captcha/security 'widget' from loading?

j16sdiz 3y ago

> Worse yet, I know that Cloudflare knows I have those certificates. Why? Because it asked for them! It doesn’t make sense for Cloudflare to request any client certificates. I think there are real bugs somewhere.

ArchOversight 3y ago
Cloudflare allows you to enable mTLS for websites: https://developers.cloudflare.com/ssl/client-certificates/enable-mtls https://developers.cloudflare.com/ssl/client-certificates/en... https://developers.cloudflare.com/cloudflare-one/identity/devices/access-integrations/mutual-tls-authentication/ https://developers.cloudflare.com/cloudflare-one/identity/de... This would then require Cloudflare to request a client certificate. This is great for securing websites using corporate identity that is derived from AD certs for example to make sure the device being used has a valid cert on it. Alongside MDM for example forcing the certificate to have a short lifespan (my $CORP uses 7 days) you can validate that the device has the correct security posture to access the resources. If for example I let my device not update the version of macOS often enough my cert expires and I can't access internal resources until I update my OS and MDM software checks that and provisions me a new device certificate.
Operyl 3y ago
It's requesting client certs for their internal intranet stuff hosted behind cloudflare.

miyuru 3y ago

I cannot access flyertalk.com, which hosts lot of useful airline content from any IP from my country. I tried reaching out via email as mentioned in the error page and admin does even have a valid email posted anywhere. I know cloudflare is not to blame here, but they provide way easy access to blocking to bad admins.

Dwedit 3y ago

The big problem I have with Cloudflare's integrity check is that all the spam domains use a fake version which mimics it, and tries to trick you into completing a captcha.

superkuh 3y ago

This has been my experience for a handful of years but of course it's getting worse. In the past I'd just be getting blocked from access commercial websites or applications and things I didn't really need. But in the last few years many scientific publishers have put all their content behind cloudflare walls. Pretty much my only hope of being able to read a paper these days is that it came out long enough ago to be on sci-hub or they published the pre-print on arxiv/bioarxiv/etc. Once arxiv goes behind a cloudflare I don't know what I'll do.

tracker1 3y ago

For good or bad, this is why I have the Privacy Pass extension installed.

gsich 3y ago

"Checking if the site connection is secure" what a blatant lie. If you can read this you already have a TLS connection.

adammartinetti 3y ago

Hi there, I'm the PM for Cloudflare's challenge platform. I'd love to look into what the cause of the problem is, so you don't see these difficulties. > Cloudflare detected the high frequency of requests and denials (but not their faulty loop that caused this pattern of requests, of course), and tagged my browser as suspicious. I can tell you at least that we don't penalize users for this looping behavior, so this wouldn't cause us to see your browser as suspicious. I hope we can dig into this more and uncover the cause of the problem. Personally, I'm a big Firefox user, and this isn't behavior I see. If there were a widespread Firefox wide issue, automated alerts would trigger and we'd consider this a critical incident. You can drop me an email at amartinetti at cloudflare if you're interested in troubleshooting.

bradly 3y ago
Anecdotaly... I use Firefox and have noticed the Cloudflare interception pages verifying I'm human appearing more often recently. Usually it is all automatic and isn't a big deal, but I have noticed a increase in how often I see these the past week.
- deleted 3y ago
  [deleted]
- stjohnswarts 3y ago
  I use a VPN 99% of the time and I definitely see more cloudflare on my systems (as opposed to when I'm bored and surfing in the library) and see the "checking" screen. It does seem to be a lot better than last year though when it seemed I was getting a captcha every time I turned around
  - JakeAl 3y ago
    I have the same issue and often just reconnecting using a different node/city resolves it Annoying as hell, but sometimes the problem is the nodes in the US show as being in EU for some reason.
- lovegrenoble 3y ago

bradley13 3y ago

Anecdote: For my programming classes, one example I use is a simple browser. It doesn't do CSS or Javacript, so display is primitive, but it works. On some sites. Many sites, especially the big ones, see that it's an unknown browser, and refuse to send content. Probably they think it's a bot. But even if it were, what's wrong with bots, as long as they're well-behaved? What kind of closed web have we let the megacorps build?

unmole 3y ago
Playing the devil's advocate: Why shouldn't a server get to decide which clients it wants to talk to?
- userbinator 3y ago
  Why shouldn't a store get to decide the $protected_class of which customers it will do business with?
  - unmole 3y ago
    User Agent is not a protected class. Neither is Intentionally Obtuse for that matter.
- 3LazTjBv-f 3y ago
  In principle? Of course. I mean I remember blocking Yandex bots from hammering some e-commerce site on shoestring budget. But each person developing a web scrapping bot realizes at most after a week that being honest with User-agent has negative impact on how well it works, and changing it to existing browser takes literally seconds.
- doctor_radium 3y ago
  Long term, I can imagine this becoming a corollary to Net Neutrality. Whenever that finally becomes a thing, the next step could be a law/rule that "public web sites need to be accessible by the public". Not that developers need to test their work in dozens of different browsers, but that they can't actively choose their customers.

krono 3y ago

The amount of times Cloudflare is making me sit through their 15 to 30 second "checking your connection" page is insane. For people going through life with ADHD such as myself, the impact of all these delays and disruptions throughout the day can be severe. Despite being properly medicated this measure is absolutely debilitating and makes for a dreadful and very taxing online experience.

jeroenhd 3y ago
Cloudflare's Privacy Pass may help here: https://privacypass.github.io/ https://privacypass.github.io/ It should significantly reduce the amount of CAPTCHAs you see in a way that's not terrible for privacy. For Safari, you can enable Private Access Tokens: https://blog.cloudflare.com/how-to-enable-private-access-tokens-in-ios-16-and-stop-seeing-captchas/ https://blog.cloudflare.com/how-to-enable-private-access-tok... Both of these mechanisms are similar to Google's web DRM proposal in that they rely on external issuers to generate tokens, but unlike Google's attempt they don't guarantee that ad blockers are disabled on pages that try to use tokens.
- tomxor 3y ago
  Wow, that doesn't sound like a terrible idea! Which is honestly surprising in this area where it feels like privacy, anonymity and human verification are incompatible with each other. I am trying to minimise my time wasted by websites, which is hard to balance with privacy, one other one is the repetitive consent forms (if you don't retain cookies, it's a never ending process). I think consent forms and human verification are the 2 biggest human time wasters.
  - jeroenhd 3y ago
    > Which is honestly surprising in this area where it feels like privacy, anonymity and human verification are incompatible with each other. The thing is, it still allows for some correlation between attestation provider and the websites themselves, potentially exposing part of your browsing history to these companies based on how many tokens you use and what websites consume them. That doesn't matter much for Cloudflare's implementation (now Cloudflare knows when you visit Cloudflare, oh no!) but with Apple's attestation provider the risks increase. The smaller the attestation provider gets or the fewer parties trust that particular attestation provider, the higher the risk becomes. It's better for your privacy than the current norm (de-anonimisation through fingerprinting while you fill out a CAPTCHA) but it's still not great. It also allows for attestation providers (and their algorithms) to arbitrarily deny you access to the web if other websites decide to start using them. Privacy in exchange for power, I'm not so sure about that. I imagine for someone suffering from ADHD the small risk that Cloudflare decides to screw you in particular is worth the massive improvement in browsing experience, but everyone will have to determine the pros and cons for themselves.

dcow 3y ago

So many privacy nuts use Chrome and don't realize this: > What about Google Chrome? > I tried all of the above in Firefox. So I naturally tried to access the same page in Google Chrome to see if I’d still be blocked. Thankfully, I wasn’t. > But of course I wasn’t because Chrome doesn’t have the same privacy- and security-enhancing designs that Firefox does. Chrome will happily collect as much private information about me and my browsing history and share them with select parties, as needed. It also doesn’t resist fingerprinting or let me modify settings to the same degree that Firefox does because Chrome relies on those fingerprinting technologies to ensure that I am targeted by ads it deems necessary for me to see. > Being blocked on Firefox and not blocked on Chrome also tells me that Cloudflare is blocking me based on the fingerprint (or lackthereof) of my browser. Everything about my connection is identical between the two requests, aside from the browser being used. It’s the same security certificates, same corporate VPN, same machine, even the same timeframe when I try to access the site. If you care about anything these days, don't use Chrome.

afavour 3y ago
I’m no Google fanboy but I wasn’t satisfied with this: > Chrome will happily collect as much private information about me and my browsing history and share them with select parties, as needed What information does Chrome provide in this scenario that Firefox doesn’t? It feels like backward logic: it worked in Chrome therefore it must be because Chrome gave extra info. In reality it could be a whole bunch of things, something as mundane as Firefox being a rarer user agent so subject to more filtering. It strikes me that all of this is an inexact science. I've run into rate limit messages with sites before now that go away when I switch browsers, no matter what the browser is. I assume it's because, with the limited information given, the DDOS protection software assumes that same IP + different UA = different computer. I have no clue but I wasn’t persuaded that this specific scenario works with Chrome because it was giving away more information. At a bare minimum at least try a third browser!
- deadbunny 3y ago
  You're conflating a downside of using Chrome and the reason they think Cloudflare blocked them.

ricardo81 3y ago

Boils down to gatekeepers doesn't it. Unfortunately there's also bad actors on the web (and the definition of bad varies). I understand reasons to try centralise the removal of that so called bad, but obviously a central group deciding on the 'bad' just isn't democratic. Ironically when chatgpt mentioned their UA on a web page the other day, users were presented with an anti-bot challenge.

Animats 3y ago

Suing Cloudflare for interference with contract[1] might be an option. Cloudflare is not protected against lawsuits by some EULA, because the outside user has no contract with them. They're a third party in the middle. Talk to a lawyer. Most contract law lawsuits are settled out of court. The great advantage of suing someone is that you get past the low-level customer support people and talk to someone who's authorized to settle. [1] https://www.lodhs.com/blog/interference-with-contractual-or-advantageous-relationships/ https://www.lodhs.com/blog/interference-with-contractual-or-...

HumanOstrich 3y ago
Can you provide examples of people suing Cloudflare to get around being blocked from accessing certain websites?

rejectfinite 3y ago

Using something like Edge (for work), Vivaldi or Brave would be better than Chrome and probably let you in instead of Chrome.

jeroenhd 3y ago

Some comments I have on this post: > Worse yet, I know that Cloudflare knows I have those certificates. Why? Because it asked for them! Not really. Cloudflare notices your browser has TLS authentication available and asks you for it. That's really annoying, but part of the protocol spec. Your browser won't send this information unless you pick a certificate and hit OK. Disable your ad blocker and you'll find that many trackers will also ask you to identify yourself this way. It's really annoying, browsers need to design better UX for this type of authentication. > · MAC address of my machine that I have previously used to access this site How does it gather your MAC address? Did you disable IPv6 Privacy Extensions? Unless the website is sitting behind the same switch as your computer or you run some kind of native application that sends the MAC address, websites can't read the MAC of your network interface. Enable the MAC randomisation that's present (sometimes even turned on by default!) in every modern OS if you consider the local switch or WiFi network to be a privacy risk. > Will I be able to create and sync these passkeys myself? Yes, assuming they follow the standard > Can only certain types of software use passkeys? If so, who decides what software meets this standard? I don't really understand the question. Any software supporting passkeys will be able to prompt you for generating or using a passkey. > Will I only be able to generate passkeys on a device with specific hardware/software requirements like a TPM, DeviceCheck, or Integrity API? According to the spec, keys can be stored in software no trouble. Websites and apps can ask for securely generated keys, but I don't think those are all that common. Hardware can also be faked relatively easily in most circumstances. > Can I, at any time, export my passkeys from one service provider and switch to another provider? Ask your service provider for export options. Most likely, you can't just dump the keys and import them elsewhere (that would defeat the point). > If a passkey is invovled in a suspicious event, will that suspicious mark propogate to any other device that uses that same passkey? Do devices that contain suspicious passkeys also get marked as suspicious? If so, would that impact the ability of that device to access other independent websites? That depends on the software using the key for authentication. Maybe?

paradox460 3y ago
I've seen the misconception a lot; people seem to think that random websites can grab your MAC like they can get your IP address.

rubatuga 3y ago

Actually cloudflare doesn't have access to your MAC address so it's a bit more difficult to attest that you are a legitimate user.

deleted 3y ago

[deleted]

xmichael909 3y ago

Cloudflare sucks, plain and simple, no idea why anyone uses it. So many better alternatives.

aeyes 3y ago
For example? Especially when taking price into account.

koito17 3y ago

Increasingly more sites are getting stuck in a Cloudflare verification loop on my end. I use Firefox on the beta channel, and I do have a few privacy extensions and a heavily modified user.js. If you want to give in to the browser fingerprinting, I have found that enabling WebGL, enabling performance timing (wow), setting network.http.referer.XOriginTrimmingPolicy to 0, among other tweaks, helps me break out of the verification loop. In other words, if Cloudflare can't reliably fingerprint your browser, you are treated as a "bot" and denied access to a huge chunk of the web. Well, in that case, I would rather be a bot than a human. Being a human seems to be increasingly annoying nowadays :)

parasti 3y ago

Insanely enough, Cloudflare sometimes puts these pages in front of API endpoints, as if that JSON were for human eyes only.

adamgamble 3y ago

One thing that sometimes gets lost is site owners that use cloudflare have sort of global options for how paranoid they want to be, then they can make specific WAF rules that can be as granular and aggressive as they want. So at least in some cases, cloudflare gets blamed for website owners setting really aggressive rules. The effect on the end user usually looks exactly the same. Case in point, I set a waf rule that blocked all non verified bot traffic from several big datacenters (Google cloud, OVH, digital ocean, etc). That turned out to be a mistake because a lot of corporations were routing their traffic through those ASNs for some reason. Now they’re blocked. They could have gotten pissed out cloudflare, the error page looks the same, but it was really misconfiguring it.

skybrian 3y ago

Fingerprinting is probably load-bearing for captchas and other anti-fraud stuff that many Internet services and businesses depend on: https://xkcd.com/2347/ https://xkcd.com/2347/ It should be replaced with something better. Unfortunately all attempts to do something better get attacked by people who don’t realize that you can’t just get rid of it, or important things will break.

nfriedly 3y ago

> Anyone who uses a de-Googled Android phone has to go to great lengths to ensure hardware attestation is working correctly [...] or else they can’t using banking apps. I have a relatively Google'd Android running lineageOS. It passes SafteyNet on a fresh install, but even that isn't good enough for one of my banking apps (or netflix) - they both also perform a CTS Profile (Compatibility Test Suite) check and block me from using the app if they don't like what they see. I ultimately had to root the phone to be able to use my bank's app. Rooting allowed me to use a fake CTS Profile, and then because it was rooted, SafteyNet started failing and I had to install a bypass to work around that. Now everything works great, except OS updates un-root the phone and then "secure" apps stop working again. (Oh, and if you mention that you're rooted, the LineageOS folks will refuse to provide any support, even for unrelated issues. Making you choose between friendly help and a usable phone is probably the only thing I don't like about LineageOS and, to my view, the biggest break from it's CyanogenMod roots.)

amatecha 3y ago

On one of my machines I run OpenBSD with Firefox with "Strict" privacy settings and "privacy.resistFingerprinting" enabled. There are so many websites I can't access, I get a straight-up 403 Forbidden page because CloudFlare has decided I am not trustworthy. I mean like pretty big companies like DigiKey, Home Depot, Canadian Tire, etc. I simply cannot use their websites, or I can load the initial page but then the API calls that provide the functionality all fail with a 403. DigiKey did something to unblock me and I can use their site again, but I know it's just a matter of time before it happens again. It's also a frequent problem on smaller sites that are simply using CloudFlare , and I never know when I'm going to be blocked from a site arbitrarily. It's especially egregious when it's a plain old text-based site like hamuniverse.com , or a small independent vendor like digirig.net ... This is one of the things that makes it so clear to me that the web is diverging into two, one that is the "clean walled-garden capitalism web" and the continuation of the original web that was open, freely-accessible and built around sharing and knowledge.

kelnos 3y ago

Prediction: if Google manages to ram Web Environment Integrity down our throats, CloudFlare will implement it as a part of these checks.

lopkeny12ko 3y ago

Anyone else find it odd that the author's company-internal work intranet, which requires a VPN to access, is deployed behind a Cloudflare CDN? Why would anyone do this?

deleted 3y ago

[deleted]

1vuio0pswjnm7 3y ago

No problem for me accessing Gitlab without using a web browser. Moreover one can use Internet Archive, Archive.today, Google's cache, etc. to avoid SNI. The author did not specify which project he was trying to access so I picked a random one to test from /explore/projects/topics/bioinformatics. No problem accessing it without a web browser. TLS1.3. No SNI. https://one-touch-pipeline.gitlab.io/otp/ https://one-touch-pipeline.gitlab.io/otp/

samcat116 3y ago

The fact that this person thinks Cloudflare has their MAC address leads me to believe they shouldn't be speculating on the "implications for the web"

datavirtue 3y ago

Please rename to "Blocked by Firefox"

Havoc 3y ago

Yeah also noticed I could t get past certain sites recently

simple-thoughts 3y ago

I personally experience this loop all the time on different sites. I’ve completely given up - if a site loops I don’t use it and try again a few weeks later. If it’s something extremely urgent I use my mobile device which for some unknown reason never loops.

deleted 3y ago

[deleted]

nhanpq 3y ago

I feel like this these days: the right to decide if I am free to choose to use or access a service or website is not based on whether I claim to be human (in captchas tests), but based on the data people collect about me - and decide on them - something that I don't know behind invisible doors. I thought privacy was on the rise after the data leaks and irresponsibility of the big tech companies, and the public's involvement in the issue of individual privacy, but it seems like everything is still a step backwards.

issung 3y ago

I use cloudflare for hosting my sites, can I and how would I disable this functionality?

foxylad 3y ago

Off topic - but how can Cloudflare block access to an internal website when accessed by via VPN? And if CF has some kind of request verification API that the internal server is using, why would you use it for an internal resource?

doctor_radium 3y ago
[dead]

phreack 3y ago

This happened to me just a few days ago. I tried to open a link in an app, which then tried to open it with an in-app WebView. Thus, the Cloudflare captcha loop of death. I could even see a "human verification failed" string appearing after clicking on an "I am human" checkbox. Alongside the annoyance of not being able to browse, this kind of language is awful! Literally being told to my face I am not a human.

Tozen 3y ago

The problem with Cloudflare is they purposely attempt to break user privacy by dangling websites as carrots. It's deception that they are attempting to determine if the person is human or not, because often they won't even show you the CAPTCHA. Even if you do get to the CAPTCHA, sucessfully doing it usually won't give you access to the website either. So, what is the point? They want people to disable any privacy protections or push usage of browsers that have no to less privacy protections, in order to access the website they are blocking. This has nothing to do with if a user is an actual threat or bot, but is more a strategy to shape what browsers are used and destroy user privacy. Cloudflare is also very aware of the numerous and constant complaints about what they are doing, coming from users and for years. They are ignored, because they have something else in mind.

1vuio0pswjnm7 3y ago

The End of the Road for Cloudflare CAPTCHAs https://blog.cloudflare.com/end-cloudflare-captcha/ https://blog.cloudflare.com/end-cloudflare-captcha/ Discriminate against all but "major browsers". Why. https://developers.cloudflare.com/fundamentals/get-started/concepts/cloudflare-challenges/ https://developers.cloudflare.com/fundamentals/get-started/c...

fartcanister 3y ago

The same for Safari. If your website is available in only one browser, this means that you need to change the content delivery provider.

monsieurUK 3y ago

I use ungoogled chromium on my openBSD machine, and I think it's better for privacy than firefox :-).

Jaskolka 3y ago

Fuck Cloudflare

rcMgD2BwE72F 3y ago

As a Firefox mobile user, I've never been able to go past that page for ~2 years. And so I've stopped visiting websites that use that system (several per days). There's no way to report that to Cloudflare so f*ck'em.