15 ms·

It's cloudflare and parasites like them that will make the internet un-free. It's already happening, I'm either blocked or back to 1998 load times be cause of "

• lillecarl 1y ago
  I use Firefox with adblocking and some fingerprinting anti-measurements and I rarely hit their challenges. Your IP reputation must be bad. They have an addon [1] that helps you bypass Cloudflare challenges anonymously somehow, but it feels wrong to install a plugin to your browser from the ones who make your web experience worse 1: https://developers.cloudflare.com/waf/tools/privacy-pass/ https://developers.cloudflare.com/waf/tools/privacy-pass/
  • godelski 1y ago
    I'm in a pretty similar boat except I frequently hit challenges. Especially if I use a VPN (which is more trustworthy than my ISP). Ironically, I'm using Cloudflare for DoH
    • lxgr 1y ago
      I'd be surprised if Cloudflare were actually correlating DoH requests to HTTP requests following them, so I don't think that's a signal they are likely to use.
      • godelski 1y ago
        Probably not. In fact, it's probably a good sign that they are being accurate about that traffic being encrypted. But I did find it ironic
  • chrismorgan 1y ago
    > Your IP reputation must be bad. And for an extremely large number of honest users, they cannot realistically avoid this. I live in India. Mobile data and fibre are all through tainted CGNAT, and I encounter Cloudflare challenges all the time. The two fibre providers I know about use CGNAT, and I expect others do too. I did (with difficulty!) ask my ISP about getting a static IP address (having in mind maybe ditching my small VPS in favour of hosting from home), but they said ₹500/month, which is way above market rate for leasing IPv4 addresses, more than I pay for my entire VPS in fact, so it definitely doesn’t make things cheaper. And I’m sceptical that it’d have good reputation with Cloudflare even then. It’ll probably still be in a blacklisted range.
    • bombela 1y ago
      Why don't your ISPs just use IPv6?
  • henrixd 1y ago
    I'm having lots of problems with fingerprinting protection on Librewolf and ungoogled-chromium. I use uBlock Origin and JShelter extensions on both. I'm always getting "your browser is out of date" despite always having the most newest versions. Some sites like Stackexchange will work after just reloading the page. And rest of the sites usually work when I remove Javascript protection and Fingerprint detection from JShelter. Sill not all of them. So, they maybe/probably want to reliably fingerprint my browser to let me continue. If I use crappy fingerprint protection, I'm not having problems but if I actually randomize some values then sites wont work. JShelter deterministicly randomizes some values using session identifier and eTLD+1 domain as a key to avoid breaking site functionality but apparently Cloudflare is beeing really picky. Tor browser is not having these problems but it uses different strategy to protect itself from fingerprinting and doesn't randomize values but tries to have unified values across different users making identification impossible.
• MichaelZuo 1y ago
  If your on ipv6, I think they have to for ipv6 addresses… there’s just way too many bots and way too many addresses to feasibly do anything more precise. If your on ipv4 you should check whether your behind a NAT otherwise you may have gotten an address that was previously used by a bot network.
  • lxgr 1y ago
    > I think they have to for ipv6 addresses… there’s just way too many bots and way too many addresses Are you really arguing that it's legitimate to consider all IPv6 browsing traffic "suspicious"? If anything, I'd say that IPv4 is probably harder, given that NATs can hide hundreds or thousands of users behind a single IPv4 address, some of which might be malicious. > you may have gotten an address that was previously used by a bot network. Great, another "credit score" to worry about...
    • MichaelZuo 1y ago
      For a whitelist system, then by definition yes? If it’s a blacklist system, like I said I’ve not heard of any feasible solution more precise than banning huge ranges of ipv6 addresses.
      • Dylan16807 1y ago
        > For a whitelist system, then by definition yes? A whitelist system would consider all IPv4 traffic suspicious by default too. This is not an answer to why you'd be suspicious of IPv6 in particular. > I’ve not heard of any feasible solution more precise than banning huge ranges of ipv6 addresses. Handling /56s or something like that is about the same as handling individual IPv4 addresses.
        • MichaelZuo 1y ago
          > A whitelist system would consider all IPv4 traffic suspicious by default too. Based on what argument…?
• slenk 1y ago
  How is Cloudflare a parasite? I can use Cloudflare, and get their AI protection, for free. I have dozens of domains I have used with Cloudflare at one point and I haven't paid them a dime.
  • fsflover 1y ago
    They put themselves as a middle man for almost the whole Internet, collect huge usage data about everyone and block anybody who doesn't use mainstream tools: https://news.ycombinator.com/item?id=42953508 https://news.ycombinator.com/item?id=42953508 https://news.ycombinator.com/item?id=13718752 https://news.ycombinator.com/item?id=13718752 https://news.ycombinator.com/item?id=23897705 https://news.ycombinator.com/item?id=23897705 https://news.ycombinator.com/item?id=41864632 https://news.ycombinator.com/item?id=41864632 https://news.ycombinator.com/item?id=42577076 https://news.ycombinator.com/item?id=42577076
    • baq 1y ago
      valid. ...but OTOH it's their customers who want all of that and pay to get that, because the alternative is worse. rock and a hard place.
      • slenk 1y ago
        Right - do I want them getting some info from me, or do I want my IP address exposed? Besides CloudFront, which still costs money, what other option is there for semi-privacy and caching for free?
        • mattl 1y ago
          bunny.net has some options
          • deleted 1y ago
            [deleted]
          • slenk 1y ago
• rockskon 1y ago
  LLM scrapers have dramatically been increasing the cost of hosting various small websites. Without something being done, the data that these scrapers rely on would eventually no longer exist.
  • benjiro 1y ago
    I think the correct term is, that unrestricted LLM scrapers have dramatically been increasing the cost of hosting various small websites. Its not a issue when somebody does "ethical" scraping, with for instance, a 250ms delay between requests, and a active cache that checks specific pages (like news article links) to rescrape at 12 or 24h intervals. This type of scraping results in almost no pressure on the websites. The issue that i have seen, is that the more unscrupulous parties, just let their scrapers go wild, constantly rescraping again and again because the cost of scraping is extreme low. A small VM can easily push 1000's of scraps per second, let alone somebody with more dedicated resources. Actually building a "ethical" scraper involves more time, as you need to fine tune it per website. Unfortunately, this behavior is going to cost the more ethical scraper a ton, as anti-scraping efforts will increase the cost on our side.
    • Tmpod 1y ago
      The biggest issue for me is clearly masquerading their User-Agent strings. Regardless of whether they are slow and respectful crawlers, they should clearly identify themselves, provide a documentation URL and obey robots.txt. Without that, I have to play a frankly tiring game of cat and mouse, wasting my time and the time of my users (they have to put up with some form of captcha or PoW thing). I've been an active lurker in the self-hosting community and I'm definitely not alone. Nearly everyone hosting public facing websites, particularly those whose form is rather juicy for LLMs, have been facing these issues. It costs more time and money to deal with this, when applying a simple User-Agent block would be much cheaper and trivial to do and maintain. sigh
  • trollbridge 1y ago
    I use Cloudflare and edge caching, so it doesn’t really affect me, but the amount of LLM scraping of various static assets for apps I host is ridiculous. We’re talking a JavaScript file of strings to respond like “login failed”, “reset your password” just over and over again. Hundreds of fetches a day, often from what appears to be the same system.
• dceddia 1y ago
  Yep this terrifies me, 100%. We’re slowly losing the open internet and the frog is being boiled slowly enough that people are very happy to defend the rising temperature. If DDoS wasn’t a scary enough boogeyman to get people to install Cloudflare as a man-in-the-middle on all their website traffic, maybe the threat of AI scrapers will do the trick? The thing about this slow slide is it’s always defensible. Someone can always say “but I don’t want my site to be scraped, and this service is free, or even better yet, I can set up my own toll booth and collect money! They’re wonderful!” Trouble is, one day, at this rate, almost all internet traffic will be going through that same gate. And once they have literally everyone (and all their traffic)… well, internet access is an immense amount of power to wield and I can’t see a world in which it remains untainted by commercial and government interests forever. And “forever” is what’s at stake, because it’ll be near impossible to recover from once 99% of the population is happy to use one of the 3 approved browsers on the 2 approved devices (latest version only). Feels like we’re already accepting that future at an increasing rate.
  • RiverCrochet 1y ago
    The Internet is not the first global network. Before the Internet, you had the global telephone network. It, too, strangulated end users, but eventually became stagnant, overpriced, and irrelevant. Super long-term, the current Internet is not immune from this. Internet standards are about getting as complicated and quirky as the old Bell stuff that was trying to make miles of buried copper the future, and if regulatory/commercial forces freeze this stuff in place, it's going to lead to stagnation eventually. Something coming down the pike I think, for example, is that IPv4 addresses are going to get realllly expensive soon. That's going to lead to all sorts of interesting things in the Internet landscape and their applications. I'm sure we'll probably have to spend some decades in the "approved devices and browers only" world before a next wave comes.
  • BugheadTorpeda6 1y ago
    [dead]
  • mattl 1y ago
    We need a reasonable alternative to some of what Cloudflare does that can be easily installed as a package on Linux distributions without any of the following to install it. * curl | bash * Docker * Anything that smacks of cryptocurrency or other scams Just a standard repo for Debian and RHEL derived distros. Fully open source so everyone can use it. (apt/dnf install no-bad-actors) Until that exists, using Cloudflare is inevitable. It needs to be able to at least: * provide some basic security (something to check for sql injection, etc) * rate limiting * User agent blocking * IP address and ASN blocking Make it easy to set up with sensible defaults and a way to subscribe to blocklists.
• brumar 1y ago
  Correction: extract monstreous profits. When I read about the revenues associated with Reddit AI deals, I can't even imagine what could possibly be deals that cover half of the internet. Cynically speaking, it's a genious level move.
• axus 1y ago
  From the server perspective Cloudflare is solving problems and not causing problems to other servers. Analogy: locks for high-value items in grocery stores are annoying to customers, but other stores aren't being coerced by the locksmith to use them.
• nickjj 1y ago
  Yep, it's really annoying. I'm using Firefox with a normal adblocker (uBlock Origin). I get hit with a Cloudflare captcha often and that page itself takes a few seconds before I can even click the checkbox. It's probably an extra 6-7 seconds and it happens quite a few times a day. It's like calling into a billion dollar company and it taking 4 minutes to reach a human because you're forced through an automated system where you need to choose 9 things before you even have a chance to reach a human. Of course it rattles through a bunch of non-skippable stuff that isn't related to your issue for the first minute, like how much the company is there to offer excellent customer support and how much they value you. It's not about the 8 seconds or 4 minutes. It's the feeling that you're getting put into really poor experiences from companies with near-unlimited resources with no control over the situation while you slowly watch everything get worse over time. The Cloudflare situation is worse because you have no options as an end user. If a site uses it, your only option is to stop using the site and that might not be an option if they are providing you an important service you depend on. Secondly they now have a complete profile over your browsing history for any site that has CF enabled and there's not much you can do here except stop using 20% or whatever market share of the internet they have, and also do a DNS lookup for every domain you visit from an anonymous machine to see if it's a Cloudflare IP range. In case you didn't know, CF offers a partial CNAME / DNS feature where your primary DNS can be hosted anywhere and then you can proxy traffic from CF to your back-end on a per domain / sub-domain level. Basically you can't just check a site's DNS provider to see if they are on CF. You would have to check each domain and sub-domain to see if it resolves to a CF IP range which is documented here: https://www.cloudflare.com/ips-v4/# https://www.cloudflare.com/ips-v4/# and https://www.cloudflare.com/ips-v6/# https://www.cloudflare.com/ips-v6/#