9 ms·
The cause of the problem is that your software is faulty by design. 1. IP addresses are to be used for packet routing. Certainly not for assigning "behavior sc
by shiomiru 3y ago
The cause of the problem is that your software is faulty by design.
1. IP addresses are to be used for packet routing. Certainly not for assigning
"behavior scores" to users in the background. IP addresses say nothing about
your visitors, my IP address could have been a complete stranger's IP
address yesterday.
2. Deciding who can access half the web based on their TLS signature achieves
nothing in the long run except reinforce browser monopolies, and goes
completely against the spirit of the open web.
I guess now I have to use Chrome for browsing the web from home. Yes, I do
run a crawler-like bot as a hobby project, I got what I was asking for.
(Funnily enough, it still works if I just emulate Chrome's TLS signature). But
I also have friends who have done absolutely nothing of sorts (no technical
skills), and still got caught up in this latest ban wave.
Let's be honest here. Your service has likely caused millions of people
harm who one day to the other are suddenly blocked from half the WWW -
not just nerds, who can get around that one way or the other, real users
who just got unlucky and now are potentially blocked from accessing websites
required for their daily lives (welcome to the 21th century). This is not
a one time problem, it has been going on for years; this time it just came
too suddenly for too many people. And this kind of harm is a logical conclusion
to the heuristics you use for determining who can view a website.
Never mind that it's ridiculous how a single company from outside my country
has the power to decide on whether I can use the web or not. That's kind of
on website owners unconditionally giving this power to CF anyway.
Now, allow me to return to purchasing proxies from shady sources for myself,
so I can keep using Firefox. Thanks and keep up the good work.
- WheatMillington 3y agoYou're being a little dramatic. It's incredibly unlikely that millions of innocent users have been blocked, and unless you have data to the contrary you shouldn't make such a claim. You know what else is harmful to the concept of the open internet? The enormous malicious botnets and other endemic problems that require a solution like CloudFlare.
- ricardo81 3y ago> It's incredibly unlikely that millions of innocent users have been blocked Is there a 'town square' where we can talk about being presented captchas and similar things from 3rd party intermediates. I think it's incredibly likely that millions of hours have been wasted on such challenges.
- amatecha 3y agoOn that note... https://www.folklore.org/StoryView.py?project=Macintosh&story=Saving_Lives.txt&sortOrder=Sort+by+Date&topic=Inspiration https://www.folklore.org/StoryView.py?project=Macintosh&stor... "Well, let's say you can shave 10 seconds off of the boot time. Multiply that by five million users and thats 50 million seconds, every single day. Over a year, that's probably dozens of lifetimes. So if you make it boot ten seconds faster, you've saved a dozen lives. That's really worth it, don't you think?" Imagine if people still thought like this about computers and software.
- ricardo81 3y agoYes. And cookie splash screens! I admire GDPR's intention but hasn't it been a massive human time sink. Not to take away from your point, just that it's all a hindrance.
- grishka 3y agoThose can at least be blocked with ad blockers and/or disabling JS.
- rvnx 3y ago@adammartinetti : maybe you could consider developing a new product where you display a GDPR consent banner once, and then these settings apply to all Cloudflare-proxied websites (by passing this consent information as an additional header to the proxied site)
- meltedcapacitor 3y ago
- thaumaturgy 3y agoData point of N+1, but I haven't been able to place online orders at Petco for about a year now because they use some Cloudflare feature that hates my browser + home internet connection. Other Cloudflare-proxied sites seem unaffected, and I'm not doing any botting/crawling, nor do I have any IoT devices on my home network. There's not enough information provided to be able to do any substantive troubleshooting. This became irritating enough that it caused two side effects: (a) I stopped shopping at Petco, and (b) I moved a pile of sites off of Cloudflare and stopped recommending them, and now sometimes recommend against them. Cloudflare is still a good, quick, cheap option for sites that receive unusual volumes of malicious traffic, so I'll still recommend them as a solution to some problems. But, they're not a good default.
- sophacles 3y agoSo you're mad at Cloudflare because Petco enabled a feature that blocked you? If Petco had developed something in-house that blocked you, would you be mad at the compiler?
- thaumaturgy 3y ago...no, I've changed my recommendations for Cloudflare because it may prevent ordinary users from using a site, and insufficient information is provided for troubleshooting purposes, and those users are likely not going to go to extraordinary lengths to report the problem. Even if they do report it, the site won't be able to troubleshoot it either. So, if you don't need it, you're probably better off without it.
- danShumway 3y agoCloudflare offers this service. If Cloudflare offered a service that enabled Petco to do something amazing would you be grateful to Cloudflare? If Cloudflare advertised on its homepage about blocking a DDOS attack on a website would you say, "meh, Cloudflare wasn't responsible for blocking that attack, they only provided a feature. The website blocked the attack."? If not, then why should Cloudflare be immune from criticism when the opposite happens? Cloudflare offered Petco the features to do this as a product and makes money off of Petco's usage of those features. I do sympathize with the perspective that ultimately tools need to be somewhat neutral and it can be dangerous to forward around responsibility. But "tools are neutral" can also be taken to an absurd degree. This isn't 5 levels of indirection here and it's not Petco going and installing a neutral piece software that they downloaded from Github. Petco is a client. They're turning on toggles that Cloudflare built into their user interface and advertises as features. There's some level of moral accountability there for how those features are abused. I'm not saying it should be illegal, I'm not saying it shouldn't be allowed, but Cloudflare is definitely at least eligible for criticism. This is a product, it's not Petco abusing Cloudflare's infrastructure; they're using the product as intended and advertised.
- waithuh 3y agoI dont get your second point. Two things can be harmful to the open web at once. CloudFlare is definitely not taking the right approach at it, which damages the open web alongside botnets. Also, botnet owners are for some reason extraordinarily nerdy and smart so they probably will find a way to fool CF every other month. Its a cat and mouse game for them while actively harming everyone else both with their botnets and the increased aggressiveness of CF caused by their incorrect solution
- ipaddr 3y agoFor every one user that makes their way on here and finds and posts here on this thread probably represent 1,000,000 plus normal users An open web is open for everyone/thing not just classes of beings you select. Bots and users can both be malicious and both can be positive.
- tenacious_tuna 3y agoI agree with the premise that most people don't know how to identity or visibly complain about a given technical problem, and so an HN thread with N anecdotes about the problem likely corresponds to N * F actual amount of real-world incidents, for some value of F > 1.... but claiming it's a factor of a million without any backing evidence is absolutely an overreach. > An open web is open for everyone/thing not just classes of beings you select. Bots and users can both be malicious and both can be positive. This I agree with. I run an archiver ~monthly on a subset of my month's browsing history, and I'd hate if that got me blacklisted from Cloudflare-backed sites for a benign purpose. (See also the idea of remote attestation)
- computerfriend 3y agoThat's a pretty good idea. Do you randomly sample, or just exclude some domains? Is there some tool out there that does it for you?
- tenacious_tuna 3y agoAssembling the list of links to archive is a manual process--I just log them in an Obsidian notebook with a category and summary, and I later post it to my blog. (I don't really think other people care, it's more for me to be able to find past things I've found interesting.) For the archival process I use ArchiveBox[1] running as a container on my NAS; I just grep through the note for `http|https` and feed the resulting list to the archiver. For everything not-hackernews I set the depth to 1, but for HN threads I do 2 so I grab whatever people may have linked in the comments. I think there's ways to hook into like, ALL Firefox history or saved posts on reddit, but that's way heavier than what I care for. [1]: https://archivebox.io/ https://archivebox.io/
- bombolo 3y agocloudflare blocks me every time I open an incognito window. No VPN, just having no cookie towards a domain automatically means I'm a bot…
- dontupvoteme 3y ago>You know what else is harmful to the concept of the open internet? The enormous malicious botnets and other endemic problems that require a solution like CloudFlare. You know what's infinitely worse? Monopolies. Half these problems can be fixed by banning certain parts of the world. It's just politically shifted out of the Overton window to do that so CF profits greatly.
- boneitis 3y agoRight. It feels silly to point out specific things when just about everything about these verification checks deployed by every megacorp throws you into a universe of suffering. If there's a place to start, it would be with eliminating the infinite challenge loops. Bad enough that IP blocks get outright blocked. Bad enough that I have to decide whether or not that blurred sliver of the edge of the wheel+shadow constitutes being part of the bicycle. Not to mention the humanitarian betrayal of the absolute highest form to farm the free human labor to train AI models when they are simply trying to browse the $#@%ing internet.
- shadowgovt 3y agoYou're going by the specified, designed use cases of those technologies. Every spec is a three-edged sword: the spec, the intent of the spec, and the use of the spec in the wild. In practice, Cloudflare does a pretty good job on far-more-than average of gluing together some heuristics in an unspec'd way to filter traffic. It sucks because you can't plan around it, but that's rather the point because the malicious actors are trying to plan around it also. (ETA: Hacker News rate-limited this post. In theory, I could have set up a sock-puppet to try and work around that, but then they would catch that too and I'd be out two accounts. So I just waited out the limit. Measure and counter-measure. ;) ).
- dontupvoteme 3y agoWhy does HN have such aggressive and seemingly illogical post rate limits anyway? Is it a theory about increasing quality of communications? It can't be a tech bottleneck.
- shadowgovt 3y agoI don't have this information first hand, but that's my assumption. Dang was handing them out like candy on January 6th. And I think he was justified in doing so; there was a coup in progress in the United States, so discourse here went completely off the rails. But it's a very easy to implement method of throttling volume, which helps improve the conversation by minimizing opportunities for people to gish-gallop. You can email and ask to have it removed; I have refrained from doing so because it serves as a gentle reminder not to get dragged down in the lowest common denominator of what passes for discourse on the site from time to time.
- dcow 3y agoI sympathize with your frustration, but you also have to admit that Cloudflare is tasked with an impossible problem: from a sea of requests, identify those that are coming from robots that are disguised as humans. So there is no perfect solution. You can't use strong identity because a user can share their identity with a robot. You have to use a crapy heuristic that only works most of the time (or tell site owners it's an application layer problem and use this SASS solution to solve the problem). I mean you admitted that you run a crawler. Cloudflare has detected that you run a crawler and has wants you to prove that you're human to access sites on their network. It actually sounds like their product worked. In any event, there should probably be better regulation around how this blocking is handled so that users aren't being unjustly blocked. If you want to run a crawler, how do you do it ethically so that you aren't targeted and your traffic blocked? If Cloudflare blocks you from accessing one site should that block extend across their whole network? How long should it last? How do you appeal the block if Cloudflare's heuristics falsely block you? If you're in a life and death situation and need immediate access to medical information and Cloudflare unjustly blocks your access and it causes harm, who's at fault? Etc.
- ipaddr 3y agoWhy are humans only allowed and shouldn't we be proactive and accept robots as equals now. We have a history of prejudice against groups and we seem clueless that we are heading their again.
- dcow 3y agoOf course I'd agree that if a robot is following the rules and behaving indistinguishably from a human but maybe just a little more quickly, then it shouldn't be pre-judged (and our detection should accommodate). But here we're talking about robots without agency being e.g. used in botnets to abuse services, or otherwise not following the rules.
- ipaddr 3y agoAll clients follow the rules if you enforce them. Break rate limit and get a timeout. Settle your payment before you send the product using bitcoin instead of Visa which is not able to do this.
- lambda 3y agoAnd even when it doesn't block you completely, it delays website loading, makes you jump through frustrating captchas, etc. It's probably third in the list of frustrating web behaviors in the past couple of years (behind GDPR popups and registration/paywalls that seem to have gotten much worse recently). And somehow there are some sites that I get CF delay walls on every time I visit. This feature is utterly broken for a good web experience; it pushes users away from sites which use it. Every time that "checking your browser" page comes up for a legitimate user should be considered a failure. Sure, it can maybe happen a few times in a thousand, but the feature is utterly broken if it comes up every time I visit the same site from the same browser not in private mode.
- cutler 3y agoWhy not take a screenshot of the CF error and send it to the website owner? It would freak me out if I thought a significant number of my website's users were being blocked by CF.
- brigade 3y agoThe work needed to maybe get it past outsourced customer support is not at all worth the effort for any site I don’t actually need to use
- simple-thoughts 3y agoI’ve done this before, and the response is always “this is the first time I’ve seen this” and “you must be a bot operator”.
- doctor_radium 3y ago+1 to anybody who creates a site to name and shame CF customers who block legitimate traffic. For a few months now I've been taking screenshots every time this happens, but with no end goal. Complaining to the individual site owners feels like a lifetime commitment, and there are virtually none I need that badly.
- janc_ 3y agoHow do you send them that when you can't access the contact form and/or contact information on their site because Cloudflare blocks it? (assuming a normal visitor, not someone who knows about whois etc.)
- NicoJuicy 3y agoThis is a very nasty comment. I was wondering if I could find some things that could lead to an exception. But I'm pretty sure that millions of users aren't using stuff like w3m pager ( https://news.ycombinator.com/item?id=34175754 https://news.ycombinator.com/item?id=34175754 ) We're all technical here, we are the edge cases. We use exotic software / combos. Let's not get carried away here The PM of cloudflare uses Firefox, I sometimes use Firefox and I don't notice any difference ( concerning this use-case at least). If you want help, perhaps describe the actual use-case that is blocking you to him. He shared his email. - country - software ( VPN, ... ) - browser - OS - traceid - ... Either way, buying shady proxies as you mentioned is already a warning flag. While using Firefox is not :)
- samcat116 3y ago> Let's be honest here. Your service has likely caused millions of people harm who one day to the other are suddenly blocked from half the WWW If this was true, Cloudflare wouldn't be a good product used by a lot of sites.
- lmm 3y agoExcluding people who are poor, weird, privacy-conscious or otherwise inconvenient from your site is a feature not a bug, especially when you can pretend it's an accident.
- bombolo 3y agoYou're assuming website owners are aware of the issue… How would they know? Cloudflare is just telling them it blocked a bunch of bots.
- tick_tock_tick 3y agoThey have before and after analytics.....
- bombolo 3y agoThey will presume the before traffic was bots? Unless they also see a drop in sales or ads they won't notice.
- tick_tock_tick 3y agoI mean if sales didn't drop why would they care?
- bombolo 3y agoIf it's a ebsite that doesn't sell anything they won't notice.
- psd1 3y ago
- hombre_fatal 3y agoYour alt solution is what?Everyone should build their shit to handle millions TB/s of DoS traffic?
- dontupvoteme 3y agoBlock the countries it comes from?
- hombre_fatal 3y agoIt sure would be nice if there was a reason DoS could only come from countries other than those of your customers/users. But that's not the case.
- creatonez 3y agoThis is what Cloudflare already does, and it's hellish for users.