12 ms·

Ask HN: Does Cloudflare block HN comments if you have code blocks in a reply?

311 points by zikduruqe 3y ago

I was going to make a comment about using netcat to send files, and Cloudflare blocks the submission. I have never seen this happen to me on HN. https://imgur.com/a/Pyhcssl https://imgur.com/a/Pyhcssl Has anyone run into this before?

ShadowBanThis01 3y ago
[flagged]
- DonHopkins 3y ago
  [flagged]
bhaney 3y ago
Any code including netcat (for it's tendency to be used in reverse shells) or SQL (for it's tendency to be used in SQL injections) tends to be blocked across the entire cloudflare-net these days.
- zikduruqe 3y ago
  Interesting. That's the first time I have run across this. Makes sense.
  - Jabrov 3y ago
    How does that make sense?
    - kevindamm 3y ago
      I think they meant "corresponds with behavior observed" not "sounds rational."
  - crotchfire 3y ago
    That does not make sense at all; it is batshit-crazy broken.
    - r1ch 3y ago
      Welcome to the world of WAFs. Pattern matching scary strings is big business.
      - cubefox 3y ago
        For the uninitiated: https://www.cloudflare.com/application-services/products/waf/ https://www.cloudflare.com/application-services/products/waf...
        wslh 3y ago
        Believe me you can find them present in popular sites you use everyday ;-) I caught one of them and reported it.
        rrr_oh_man 3y ago
        Thanks. For me WAF is https://en.wikipedia.org/wiki/Wife_acceptance_factor https://en.wikipedia.org/wiki/Wife_acceptance_factor since the 2000's era of German audio enthusiast forums
      - refulgentis 3y ago
        what dark magic is this? do you put it in front of all POSTs or something?
        Uvix 3y ago
        Not just POSTs, but any HTTP request - headers & query strings get inspected too.
    - josephcsible 3y ago
      Indeed. WAFs need to die; they're basically all just doing <https://thedailywtf.com/articles/Injection_Rejection https://thedailywtf.com/articles/Injection_Rejection>.
      - bandergirl 3y ago
- tambourine_man 3y ago
  In a site like HN, that’s ridiculous.
  - goles 3y ago
    See: https://news.ycombinator.com/item?id=38939668 https://news.ycombinator.com/item?id=38939668
    - Dylan16807 3y ago
      I see it. It explains why cloudflare is involved at all. But I don't see how this is related to the comment you replied to, which is saying that the text filter is ridiculous.
  - renonce 3y ago
    Sites like HN could have disabled WAF. It's entirely configurable on HN's side. Let's just wait until dang wakes up and implement the required changes.
    - cwillu 3y ago
      “Thanks for the heads-up! We'll probably be off Cloudflare fairly soon - I think that's more likely the better fix. But if we end up being forced to stay on it, we'll look into configuring those rules.” --reply to an email I sent
- creatonez 3y ago
  AFAIK most of these filters are disabled by default when setting up your website on Cloudflare, so most websites using the Cloudflare network likely have this turned off.
  - stingraycharles 3y ago
    This is correct, it’s part of their WAF offering where there is rule-based blocking of content.
    - judge2020 3y ago
      And requires the $20/month plan to enable.
    - viraptor 3y ago
      Also, it's not one magic switch. You can switch each of those rules on/off. Ideally HN would allow most of the injection ones, otherwise we won't be able to post examples of specific SQL patterns and the like.
- deleted 3y ago
  [deleted]
- dvh 3y ago
  drop table users where id=id
- kristianp 3y ago
  Select id, email from users; Works. Any sql thats actually blocked?
gus_massa 3y ago
To get a fast answer, it's beter to send an email to the mods hn@ycombinator.com
- LeoPanthera 3y ago
  In this case, I appreciate that OP made a public post, in case others encounter the same problem.
  - gus_massa 3y ago
    dang is not reading every single post in real tieme, so posts may get unoticed and the guidelines ask to send an email instead. In this case, it looks like some comments are giving good advice about the tradeoff and how to fix it, so I have to agree with you.
kaimac 3y ago
Cloudflare has access to everyone's cleartext? I was unaware of this. NSA must love that
- strombofulous 3y ago
  Yes, although it requires configuration https://developers.cloudflare.com/ssl/get-started/ https://developers.cloudflare.com/ssl/get-started/
  - arch-choot 3y ago
    I'm pretty sure the default is they can see all the cleartext, since their product is based on TLS interception, for example to evaluate page rules. This is also how they insert extra headers in both the request and response.
  - creatonez 3y ago
    When you add a DNS rule, it's configured as proxied by default. Here is what it looks like in the UI: https://i.imgur.com/TO2Tfk3.png https://i.imgur.com/TO2Tfk3.png https://i.imgur.com/jVW5db4.png https://i.imgur.com/jVW5db4.png
  - midasuni 3y ago
    If cloudflare have thr certificate’s private key and are advertising the A record they have access to everything you send, from emails to credit card numbers.
- ytch 3y ago
  It's a TLS termination proxy that decrypt and re-encrypt your TLS packet. Technically Cloudflare can read anything unless you add your own crypt layer on top of TLS.
- deleted 3y ago
  [deleted]
- judge2020 3y ago
  Same for Akamai, Cloudfront, Fastly, etc. Pretty much every business that wants to offload DDOS protection, caching,and some level of frontline security uses a proxying CDN. An alternative is to keep all of your CDN assets on a CDN bucket on its own hostname, with your main secret-containing business apps on your own servers, but it costs a lot to manage this level of separation and the payoff is only protection against the theoretical attack of "NSA can't attack our users/spy on them". If the NSA ever did do this on a large enough scale or to target a particularly notable person, it's very unlikely it would be kept a secret for long, and the end-business that used Cloudflare et al. wouldn't be implicated whatsoever since every business uses one of the big CDN providers.
crotchfire 3y ago
[flagged]
- throwaway14356 3y ago
  finally a way out
- ShadowBanThis01 3y ago
  [flagged]
  - Dylan16807 3y ago
    I hope you figure out that annoying people doesn't make you right. We can only dream of a world where it's difficult to be annoying, and requires putting in some effort to be right first. > stealing from users What a weird definition of stealing. And it's as much an issue with your browser if hitting back doesn't return the text. There are extensions to improve that behavior. But what I find really interesting is that you seem to think being mad about an issue is a reason to break a completely unrelated rule?
    - ShadowBanThis01 3y ago
      [flagged]
      - Dylan16807 3y ago
        I didn't say to avoid jokes. Joking and being antagonistic are different things! And I still don't think stealing is the right word for that kind of technical issue, especially when it's still half your browser's fault. > which you didn’t manage to specify Why would I need to specify something you brought up? "OH NOES comparing HN to Reddit violates "policy."" > and being “mad…” Is "aggressive griping about" better? People usually simplify that to "mad about". > One of them has gone through and downvoted all my posts now Almost every post you made inside that 24 hour downvote window deserves it, so depending on how literal that "all" is, they're probably helping and not a bad actor.
furyofantares 3y ago
Huh. Just tried submitting the same comment with the same result. Minimal test, if I try to edit this post removing the asterisk, I get the "banned" page nc* 192.168.2.100
- deleted 3y ago
  [deleted]
neilv 3y ago
Probably not related, but I've been getting lots of throttling-like huge page load delays on HN the last couple days, only when logged in. Any idea whether that's just an overloaded application server, or something Cloudflare is doing?
- deleted 3y ago
  [deleted]
- pixelesque 3y ago
  That's often the case with HN I think from past experience when there are large threads on HN, and dang has in the past said that's due to the application server.
  - keepamovin 3y ago
    Single core application server no less
    - diggan 3y ago
      Crafted by first creating a language, then a framework and then the "news" application. The holy grail of NIH :)
      - keepamovin 3y ago
        Do things that don't scale! haha
djha-skin 3y ago
Related: https://news.ycombinator.com/item?id=38966035 https://news.ycombinator.com/item?id=38966035
- deleted 3y ago
  [deleted]
hyperhello 3y ago
What if you Base64 encode this? Pretty trivial to add to the form logic.
- dylan604 3y ago
  yeah, i'd expect dang to just jump right on that. just because you feel it is trivial does not mean that it should be done.
- kevincox 3y ago
  That's how one of my past employers resolved this. Basically base64 encoded every field in the JSON as someone reported a bug where the WAF blocked it. Not only was this done inconsistently and was super tedious but completely defeated the purpose of the WAF. (Except of course to check the checkbox that we had a WAF.)
usr1106 3y ago
So HN uses Cloudflare? That surprises me because typically I notice sites using Cloudflare because my mobile running GNU Linux cannot pass their dreaded Turnstyle. Luckily that does not happen for HN.
- judge2020 3y ago
  They used to run it but stopped (I want to say) around 2016 or 2017. Another poster here linked[0] to how dang confirmed it is to protect against a DDOS attack. 0: https://news.ycombinator.com/item?id=38939668 https://news.ycombinator.com/item?id=38939668
  - usr1106 3y ago
    The cert is Cloudflare today.
    - judge2020 3y ago
      I meant to imply that Dang only re-added CF recently due to the attacks. They haven't used CF in many years to my knowledge.
      - redcobra762 3y ago
        Cert was issued over 9 months ago.
        ffpip 3y ago
        The certificate is for *.ycombinator.com. The main YCombinator site might have been using Cloudflare seen years, and now HN might have been added to the same account to protect from a DOS attack. So might be the same cert?
    - nuker 3y ago
      It also means Cloudflare sees HN traffic in plaintext, including login/passwords
zaik 3y ago
[flagged]
nneonneo 3y ago
I ran into this when trying to post a comment with ../ ../ ../ etc/ passwd (remove the spaces)
- M4v3R 3y ago
  Yep, confirmed on my side too that it does indeed get blocked.
deleted 3y ago
[deleted]
jrockway 3y ago
This line works: nc -l -p 1234 -q 1 > testfile.txt < /dev/null The other one doesn't. alias foobar=nc cat testfile.txt | foobar 192.168.2.100 1234 I was hoping that it was a "useless use of cat" filter, but nope. It just doesn't like the bytes nc next to an IPv4 address. This is also fine, but blocked if you change the slash to a dot: nc 192/168.2.100 1234 This works too: nc \ 192.168.2.100 1234 OK, that's all for now. Can you believe people pay money for "web application firewalls"?
- david_draco 3y ago
  Good that there are open source alternatives: socat - TCP4:192.168.2.100:1234 socat TCP4-LISTEN:1234,fork,reuseaddr - https://www.redhat.com/sysadmin/getting-started-socat https://www.redhat.com/sysadmin/getting-started-socat
  - deleted 3y ago
    [deleted]
- gnfargbl 3y ago
  > Can you believe people pay money for "web application firewalls"? I think it's like a lot of things in computer security, in that system owners just don't want to be the slowest gazelle in the herd. If an attacker is mass-exploiting some new remote vulnerability, then maybe the WAF means that you're one of the lucky ones who doesn't get hit. And yes, that's a very big maybe there. WAFs don't do much to prevent targeted attacks except to require the attacker to craft a WAF bypass. As you've shown.
  - H8crilA 3y ago
    They also have an organizational purpose. Once an attack happens you can shift the blame onto the WAF. And the WAF provider can, if needed, claim this to be a novel attack against which they have prepared for the future. Even issue an emergency patch that detects and blocks the novel backslash-newline line break technique. (I'm exaggerating here, but only a bit)
Jamie9912 3y ago
I don't remember HN being on Cloudflare. Have they recently added it?
- ignoramous 3y ago
  Yes, I see news.yc proxied through Cloudflare. Also: https://www.nslookup.io/domains/news.ycombinator.com/dns-records/ https://www.nslookup.io/domains/news.ycombinator.com/dns-rec... / https://archive.is/R9BE8 https://archive.is/R9BE8
sixhobbits 3y ago
Test `nc 192.168.2.100`
- rstuart4133 3y ago
  There is no need to do that here. Putting the text into the 'About:' box of your HN profile works just as well while not cluttering up the thread.
buro9 3y ago
Probably, the WAF, specifically Cloudflare specials, matches a number of things. And as a lot of it is just regex matching the context of where the match occurs isn't precise. Additionally cloudflare doesn't know what is safe for a given site, so it has to be a little conservative. The sites that can handle malicious input, or are tech sites that expect things that are SQL or commands that may contain directory traversal, these are in the minority. Essentially these are false positives, which are typically viewed as more acceptable than false negatives as those would allow attacks through. These things are configurable by the site owners, but the issue here is that the site owners are not shown the code of the rules, so have to guess from the names and descriptions whether something is safe to disable, meaning everyone just leaves everything enabled. Usually reporting this to a site owner with the cloudflare trace id is sufficient to enable the site owner to disable a rule that is causing false positives, as the site owner can use the cloudflare dashboard to search the trace id. I do not work there any longer (left 3 years ago), but did write significant parts of the firewall and also manage the firewall, WAF, and DDoS protection teams.
kxrm 3y ago
Apparently you can get around it using other white-space characters, here I am using a horizontal tab between nc and the IP. nc 192.168.1.100 8000
cwillu 3y ago
For reference, the id in the block message is “Cloudflare Ray ID: 845543eb88d461ee”
ifeja 3y ago
nc domain.com
6865 3y ago
aoeaoe
kevincox 3y ago
I'm sure that there is a huge chart on their Cloudflare dashboard about how many attacks were blocked! This is one thing that gets me, all of the reporting Cloudflare provides treats every block as a huge success. Nothing to help identify actually attacks vs false positives. Let also false positives that would have actually has a negative effect on the application behind the WAF.
deno 3y ago
Cloudflare is just dumb, they block XHR requests randomly in the same session they’ve already challenged breaking websites in not quite obvious ways and have been doing that for as long as I can remember. Trying to do anything on for example Montana's SOS BIZ portal takes a lot of patience. They’re like TSA of the Internet but at least with TSA you can pay for a fast pass.
ForestCritter 3y ago
cloudflare blocked me from signing in to my petflow account to buy cat food. It was in an endless verification loop. Awhile back it did the same with my paid crunchyroll subscription. I don't code, I have a very ordinary setup with a well known browser. Apparently cloudflare now owns our access to the internet and can block whom it pleases, when it pleases, no recourse. The internet is soon to be available only to those who fit cloudflare's criteria, whatever that may be, as long as companies keep buying in to the third party control.
deleted 3y ago
[deleted]
1vuio0pswjnm7 3y ago
"If you're seeing this message, that means JavaScript has been disabled on your browser, please enable JS to make Imgur work." Well, it's a client that has no Javascript engine. There is no need to use Javascript. This works for me just fine: https://i.imgur.com/YtepoDbh.jpg https://i.imgur.com/YtepoDbh.jpg
- EasyMark 3y ago
  ever since imgur started hating VPN users, I hate it back.
  - flexagoon 3y ago
    You can use Rimgo, an alternative frontend for Imgur Just replace "i.imgur.com" with one of the Rimgo instances: https://rimgo.codeberg.page/ https://rimgo.codeberg.page/ (The most popular ones are usually rate-limited though)
    - deleted 3y ago
      [deleted]
Am4TIfIsER0ppos 3y ago
How can Cloudflare be reading anything you send? Your connection is encrypted to HN's server, is it not? They don't MITM everyone's connection.
deleted 3y ago
[deleted]

bejk22 3y ago
That makes using https instead of http a lot less relevant.
- judge2020 3y ago
  https is important for preventing spying by anyone else in between you and the server. ISPs, coffee shop owners, schools, etc used to spy on http traffic to see what people were doing/searching for, and ISPs like xFinity injected code into non-https pages to show "important messages" to users, e.g. going over your bandwidth limit[0]. The only weak link now is Cloudflare, which is still "less secure than a direct connection" (with respect to government spying, bugs[0], hackers, etc) but the threat level is drastically reduced. 0: https://blog.ryankearney.com/2013/01/comcast-caught-intercepting-and-altering-your-web-traffic/ https://blog.ryankearney.com/2013/01/comcast-caught-intercep... 1: https://news.ycombinator.com/item?id=13766339 https://news.ycombinator.com/item?id=13766339
  - EasyMark 3y ago
    I too inject porn images, inverted images, backwards texst, etc in http back in the day for people piggybacking (without permission) off my wifi.
  - xk_id 3y ago
    Is there a way to know when the encrypted content i send to a site is also being provided to Cloudflare?
    - viraptor 3y ago
      Check the whois entry for the IPs that domain resolves to. If they belong to CloudFlare, they can see the plaintext traffic. Same for Akamai, Cloudfront and others.
    - tick_tock_tick 3y ago
      No, just like there is no way to know if a site not fronted by cloudflare decided to send all traffic their after.
    - tcmb 3y ago
      Isn't the SSL certificate being issued by Cloudflare a giveaway? I'm guessing it's a sufficient condition, bit not a necessary one. I.e, a could be using Cloudflare's WAF with a SSL cert issued by somebody else.
      - judge2020 3y ago
        Cloudflare can issue from Google Trust Services/Digicert with ACM[0] and often does even without ACM (although maybe only for Business/Enterprise domains). 0: https://developers.cloudflare.com/ssl/edge-certificates/advanced-certificate-manager/ https://developers.cloudflare.com/ssl/edge-certificates/adva...
    - judge2020 3y ago
      https://judge.sh/cdn-cgi/trace https://judge.sh/cdn-cgi/trace https://news.ycombinator.com/cdn-cgi/trace https://news.ycombinator.com/cdn-cgi/trace Every Cloudflare site will respond to this URI.
- anonym29 3y ago
  Hardly! Nobody is forcing you to consent to MITM, you freely choose it every time you voluntarily use a website that utilizes one.
  - anonym29 3y ago
    To downvoters: please don't shoot the messenger. I'm not happy about the existence of Cloudflare (or their competitors who do the same thing) either. That said, the choice is yours whether or not to use sites that utilize such untrustworthy MITM providers, like Cloudflare. There are even browser plugins that can automatically block connections to such untrustworthy entities. This isn't an endorsement, and you should always review the source code of any browser extensions you're utilizing due to the risks extensions themselves can pose, but I personally use one called Cloud Firewall and it works great. (https://addons.mozilla.org/en-US/firefox/addon/cloud-firewall/ https://addons.mozilla.org/en-US/firefox/addon/cloud-firewal...)
    - Dylan16807 3y ago
      An extension that tries to to block cloudflare is getting closer to making your original statement true, but it's still not true. There aren't obvious signs up front that a site is using cloudflare. Failure to spend time investigating is not "freely choosing it".
      - anonym29 3y ago
        >There aren't obvious signs up front that a site is using cloudflare. You're joking, right? It takes 2 seconds to click the padlock in your browser, click through once more, and see "Verified by: Cloudflare, Inc". You don't even need to view the certificate. If 2 seconds and 2 clicks is too much time and effort, it's obviously not actually that important to the user in question.
        Dylan16807 3y ago
        https://www.cloudflare.com/ssl/keyless-ssl/ https://www.cloudflare.com/ssl/keyless-ssl/ https://developers.cloudflare.com/ssl/edge-certificates/custom-certificates/ https://developers.cloudflare.com/ssl/edge-certificates/cust... It's not always that simple.
- cheekibreeki2 3y ago
  All of the modern http performabce optimizations require https.
bomewish 3y ago
They kept the other spying secret for a long time and it was only due to pretty heroic actions by one person that it got exposed. So I duno.

viraptor 3y ago
While this is true and worth reminding the ops about, it still sucks because many people don't understand the issues they cause by turning WAF on. CloudFlare should have a big "I understand I'll block many legit clients when I enable this" checkbox. Or you know... fix it in general. Or at least have a "report this block as invalid" link on the page.
- Jamie9912 3y ago
  Agreed, I believe the default Firewall security level is "Medium" and I think that's far too strict. First thing I do when adding a new zone is to set it to "Essentially off"
  - Aachen 3y ago
    First thing I do is not use cloudflare when I don't need big brother anyway
    - fragmede 3y ago
      Which is easy enough to say, but how do you protect your site from being ddosed?
      - Aachen 3y ago
        None of my sites have been in over a decade of hosting from a residential connection When it's needed, it's needed, but it amazes me how many people feel they need big brother protection for their personal blog and nextcloud
        deleted 3y ago
        [deleted]
        fragmede 3y ago
        how many people do you piss off with the opinions you post on your blog? enough to warrant being DDoS'd by an emotionally stunted highschooler with their parents/stolen credit card and the ability to Google for a botnet?
        Aachen 3y ago
        Almost nobody who uses big brother as an individual ever does. What would anyone care about a nextcloud login panel? Or a reasonably civil personal blog? And yet they enable cloudflare for yet another small corner of the internet :(
- aeyes 3y ago
  Cloudflare WAF doesn't block clients in general, it blocks based on the data the client sends to the server. Unless your client sends a string which matches one of the WAF patterns the site will work fine. It only blocks individual requests. Now the problem here is that you probably shouldn't enable the WAF without having it in log only mode for a while if you are operating a site which let's users submit arbitrary text input. Of course it's going to match... You'll have to adjust the configuration.
  - bombcar 3y ago
    I’ve yet to see a WAF that wasn’t eventually accidentally triggered by some zip file. I’ve had to recompress zip files with a higher compression setting to get around whatever string was triggering it.
helsinkiandrew 3y ago
I always feel the turnstyle makes a website feel a bit condescending - you need to test MY connection before I get to your crappy site? Is there a reason it needs to be visible whilst performing checks? or is it just security theatre?
- egeozcan 3y ago
  I always assumed that it is running some client side sanity checks to detect automated user agents but never checked.
  - bombcar 3y ago
    It’s just doing hash math (think like bitcoin mining) to make your CPU burn enough processing time to make a layer seven DDoS not worthwhile. It works. Because now the server uses way less processing time than the client did.
    - peetistaken 3y ago
      That is not true. It does a whole bunch of checks, like fingerprinting your GPU, environment, etc. The checks are even run in a custom VM, and are heavily protected. The gathered data is then sent back to cloudflare, and you either get an access cookie (cf_clearance) back, or not.
- explaininjs 3y ago
  There are many options to configure it, the main reason to make it always visible and blocking is that the callbacks for managing the hidden/on-demand version are wonky and can break in unexpected ways leaving your site entirely unusable, with the only indication being some errors logged to console.
- zhfliz 3y ago
  it's free advertising
- rollcat 3y ago
  > or is it just security theatre? WAF in general is security theatre. If your operation genuinely benefits from one, I dread for what's sleeping underneath. Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.

mac-chaffee 3y ago

WAFs are 2000s-era software that have long overstayed their welcome: https://www.macchaffee.com/blog/2023/wafs/ https://www.macchaffee.com/blog/2023/wafs/

mr_mitm 3y ago
Nice summary. I can add to the list of attack vectors a case where the WAF introduced a reflected cross-site scripting vulnerability. The site it was supposedly protecting was blank, i.e. it just returned a 404 error or something. But just by appending a URL parameter with JS in it, the WAF would trigger and reflect the code. So I was able to build an outlook web app lookalike for phishing on a site with the domain of the company.
leftnode 3y ago
Excellent article, thanks for writing and posting it.
kazinator 3y ago
Summary: 1. WAFs require entire requests to be buffered in order to be scanned before the server sees them. This can require lots of RAM. 2. WAFs scan requests with all sorts of hacky rules, which takes gobs of CPU time. 3. The hacky rules look for programming language syntax, for which the attackers can easily find alternative expressions to get around the rules. 4. ... yet, WAFs have high false positive rates. 5. All that kludgly processing is a security weakness. WAFs tend to be closed source behemoths written in low-level languages.
d0mine 3y ago
There are many valid points. Though it doesn't cover many things modern wafs systems do in addition to the regex rules. The question what is the alternative and the suggested alternative is that everyone become perfect security expert. It is even less likely to succeed than creating security-aimed software by professionals. Consider what is the proportion of sites that are created by people who knows zero about security. Wordpress is like on 40%+ web sites (ridiculous). WAFs unlikely to prevent targeted attacks, they don't have to to be useful. In practice, simple measures can prevent many common attacks.
oars 3y ago
Interesting article about whether WAFs are actually useful or not in modern times. Thanks!
whodev 3y ago
I was kinda with you until you made this statement: > No, "defense in depth" is not a valid excuse to use a WAF anyway, because it provides no real defense! I have to disagree here. You are making assumptions that every developer in an org will always do the correct thing and deploy code that won't be exploitable to SQL injections, XSS, file inclusion, etc... That's just not the case. I'm all for doing the correct thing, and not just performing security theater, but WAFs do offer some protection. You need multiple layers of security covering the holes that may left in other layers. And a WAF can be one of those layers of protection.