3 ms·
I run a MITM proxy for adblocking/general filtering and within the past little while I've noticed CloudFlare and other "bot protection" tends to get me blocked
by userbinator 5y ago
I run a MITM proxy for adblocking/general filtering and within the past little while I've noticed CloudFlare and other "bot protection" tends to get me blocked out of increasingly more sites I come across in search results, so this will be very useful for fixing that.
However, I should caution that in this era of companies being particularly user-hostile and authoritarian, especially Big Tech, I would be more careful with sharing stuff like this. Being forced to run JS is bad enough; profiling users based on other traits, and essentially determining if they are using "approved" software, is a dystopia we should fight strongly against. Stallman's Right To Read comes to mind as a very relevant warning story.
- oh_sigh 5y agoMaybe I'm just a techno-optimist, but I suspect big tech companies don't give a hoot about you running "unapproved" software, but rather care about their services being abused and "unapproved" software is just a useful signal that fails on a tiny percentage of total legit users.
- llampx 5y agoYou are a lot more charitable than I am. I believe the big tech companies use dark patterns to get us to sign up, improve their metrics and hoover up our data.
- saurik 5y agoThis is a distinction without a difference.
- bayindirh 5y agoJust trying to keep services operational is a fine goal to pursue as an operator, but forcing users to small inbound funnels for the service is detrimental too. There needs to be better research to be done to allow simpler ways of operation to continue working. A browser is becoming a universal agent by itself, but many people (maybe increasingly) use terminal to access to the resources, and stonewalling these paths are never OK in my book.
- cma 5y agoI've noticed even GitHub has a login wall now for comments on open source projects. They truncate them if you aren't logged in, similar to reddit on mobile, instagram, twitter, etc. Hopefully the mobile version doesn't start pushing you to install some crappy apps where you can't use features like tabbed browsing, tab sync with another machine, etc.
- caulk 5y agoThe reasoning behind that might be the myriad of scrape-and-publish SEO spam pages with GitHub content.
- raxxorrax 5y agoNot sure if I am buying that excuse. I think they want to nudge people to make accounts and login. Really shady in case of Github and many other sides that are successful because of user content in my opinion.
- userbinator 5y agoIronically, that makes the scraped copies more useful because they aren't truncated (at least for older pages) and I can actually get all the content. I wonder if that might be at least why Google seems to be giving them more weight in search result rankings.
- fake-name 5y agoCloudflare is likely one of the worst things that has happened to the internet in recent history. Like, I get the need for some protective mechanisms for interactive content/posting/etc, but there should be zero cases where a simple HTTP 200 GET requires javascript/client side crap. If they serve me a slightly stale version of the remote resource (5 minutes/whatnot) that's fine. They've effectively just turned into a google protection racket. Small/special purpose search/archive tools are just stonewalled.
- buro9 5y agoYou can't turn it off as a Cloudflare customer either. The best you've got is "essentially off" but that wording is such because even with everything disabled there are still edge cases where their security will enforce a JS challenge or CAPTCHA.
- kevincox 5y agoAt least on their basic plan there is also little to no indication of how often this is triggering. Leading to having know idea what the various settings are.
- rtpg 5y agoNot to be too dismissive of this, but for companies trying to just run a service and getting constantly bombarded by stuff like DDoS issues, Cloudflare and its ilk lets them service a large portion of "legitimate" users, compared to none. I don't really know how you resolve that absent just like... putting everything behind logins, though.
- mschuster91 5y ago> profiling users based on other traits, and essentially determining if they are using "approved" software, is a dystopia we should fight strongly against. Stallman's Right To Read comes to mind as a very relevant warning story. Right to Read indeed... fanfiction.net has over the last months become really annoying. Especially at night, when you have the FFN UI set to dark, and then out of nothing a bright white Cloudflare page appears. Or why the Cloudflare "anti bot" protection leads to an endless loop when the browser is the Android web view inside a third-party Reddit client.