7 ms·

A new Cloudflare Web Application Firewall

110 points by migueldemoura 6y ago

jgrahamc 6y ago
The plan is to open source the old WAF code (that I wrote long ago) once the new WAF is fully rolled out.
- _kyran 6y ago
  Curious if you have any stories from working on this? I'd love to hear what some of the hardest problems to solve were?
  - jgrahamc 6y ago
    You can start here: https://www.youtube.com/watch?v=nlt4XKhucS4 https://www.youtube.com/watch?v=nlt4XKhucS4
    - exikyut 6y ago
      Slides: https://github.com/cloudflare/jgc-talks/raw/master/nginx.conf/2014/cloudflare-lua-waf.pdf https://github.com/cloudflare/jgc-talks/raw/master/nginx.con... Above link is direct download, which I'm biased towards since Chrome's PDF viewer supports left and right arrow keys. Protip for users with tiny* screens: right-click video twice, enable Picture-in-Picture, arrange video so slides are still visible so you can follow along) (* Specifically <24", ie laptops)
    - exikyut 6y ago
      Couple of points from the video: 1. The regular expression simplifier (https://youtu.be/nlt4XKhucS4?t=1102 https://youtu.be/nlt4XKhucS4?t=1102) stood out as particularly interesting - I get the impression it was partly "mostly simple", and partly battle-tested/nontrivial/hand-tuned. Speaking not-entirely-rhetorically, this would probably be a very interesting tidbit to study. 2. You mentioned at https://youtu.be/nlt4XKhucS4?t=2272 https://youtu.be/nlt4XKhucS4?t=2272 in response to a question that you apparently pass PNGs and other binary content "straight through" (in the context of file upload), ie bypassing the WAF. Given things like... - webpage in JPEG (http://lcamtuf.coredump.cx/squirrel/ http://lcamtuf.coredump.cx/squirrel/, https://news.ycombinator.com/item?id=12262470 https://news.ycombinator.com/item?id=12262470, https://news.ycombinator.com/item?id=4209052 https://news.ycombinator.com/item?id=4209052), - JavaScript in EXIF (https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-ex...) - PHP in EXIF (https://web.archive.org/web/20130708132109/https://websec.io/2012/09/05/A-Silent-Threat-PHP-in-EXIF.html https://web.archive.org/web/20130708132109/https://websec.io...) - HTML+JavaScript+1021 byte demo inside PNG: https://news.ycombinator.com/item?id=24824299 https://news.ycombinator.com/item?id=24824299, http://www.p01.org/MONOSPACE/ http://www.p01.org/MONOSPACE/ (general NB: "Packed version" link under "Additional links" actually loads the demo for me in Chrome, but clicking through from HN and loading the URL directly doesn't - some sort of bizarre CORS-related thing?) ...I presume the status quo has changed somewhat here. Hearing how/what's going on in this space would be very interesting.
- anotherhue 6y ago
  Do you get to go 'off-call' then?
  - jgrahamc 6y ago
    I haven't been on call for that code for a long time. There's a whole team that works on it and has been improving it for quite a few years.
    - anotherhue 6y ago
      Glad to hear! It's a little jarring that the Web-UI throws it's hands in the air when you add a few boolean clauses so I look forward to some improvements there. Cloudflare is, in general, a delight to use.
      - jgrahamc 6y ago
        Email me (jgc) any issues.
      - alberth 6y ago
        > "Cloudflare is, in general, a delight to use." Agreed. There's lots of companies in the space that Cloudflare operates in. Cloudflare is the "macOS/Apple" of the market whereas their competition is the unwieldily mess that is "Linux".
        luckylion 6y ago
        On a technical level: I agree, it works great. Their dashboard needs some work. What I find annoying is mostly the "load everything in single ajax requests, but don't set the size of it in HTML", which makes elements jump around while stuff above them is being rendered. That's very annoying when your connection isn't great for some reason and buttons get replaced by different buttons. I get it, "use the API", but when you need to do something manually, I'd prefer a higher total load time over something that's asking for mistakes. There's a good reason Google is heavily advising to get rid of layout shifts. The other thing I took personally was them removing the "remember me" functionality in a two step process: first it was broken, and then they removed the feature alltogether it instead of fixing.
- alberth 6y ago
  Sad to see LuaJIT is no longer in use. LuaJIT was so far ahead of it's time (and still is), it's sad to see the current state of it since Mike Pall has moved on. The learned lesson for me is that, even if you have a vastly superior offering - the success of something is hugely correlated to the community behind it.
  - fsfod 6y ago
    Mike Pall is still working on LuaJIT just last week he pushed the first stage of string buffer support sponsored by some network company https://github.com/LuaJIT/LuaJIT/commit/4c6b669c419f313306b9e6ee43be4ad5f6d73ec6 https://github.com/LuaJIT/LuaJIT/commit/4c6b669c419f313306b9...
    - exikyut 6y ago
      *Activates [APPLAUSE] indicator*
- butt_hugger 6y ago
  Do you have any plans to stop ruining the open and secure web?
- exikyut 6y ago
  Wow. I was imagining the possibilities of being able to share some of the basic techniques behind the system, but being able to have an actual MWE of real racing stripes™ is cooler than I could ever have imagined would be viable. Thanks.
lovedswain 6y ago
Sorry, you don't get to market to folk who're willing to read the words "Magic Quadrant" and post that shit to HN at the same time. Pick your poison! We're a technical crowd, remember? Shudder. I wonder if the CloudFlare folk will be returning to their offices wearing suits when lockdown is over.
ChrisArchitect 6y ago
Recently for a few sites I run I realized I am increasingly verrry dependent or leaning heavily on this Firewall to keep so much Bot and spam traffic off my site.
Farahqq 6y ago
Farajqq
Farahqq 6y ago
25

110 points by migueldemoura 6y ago

jgrahamc 6y ago

The plan is to open source the old WAF code (that I wrote long ago) once the new WAF is fully rolled out.

_kyran 6y ago
Curious if you have any stories from working on this? I'd love to hear what some of the hardest problems to solve were?
- jgrahamc 6y ago
  You can start here: https://www.youtube.com/watch?v=nlt4XKhucS4 https://www.youtube.com/watch?v=nlt4XKhucS4
  - exikyut 6y ago
    Slides: https://github.com/cloudflare/jgc-talks/raw/master/nginx.conf/2014/cloudflare-lua-waf.pdf https://github.com/cloudflare/jgc-talks/raw/master/nginx.con... Above link is direct download, which I'm biased towards since Chrome's PDF viewer supports left and right arrow keys. Protip for users with tiny* screens: right-click video twice, enable Picture-in-Picture, arrange video so slides are still visible so you can follow along) (* Specifically <24", ie laptops)
  - exikyut 6y ago
    Couple of points from the video: 1. The regular expression simplifier (https://youtu.be/nlt4XKhucS4?t=1102 https://youtu.be/nlt4XKhucS4?t=1102) stood out as particularly interesting - I get the impression it was partly "mostly simple", and partly battle-tested/nontrivial/hand-tuned. Speaking not-entirely-rhetorically, this would probably be a very interesting tidbit to study. 2. You mentioned at https://youtu.be/nlt4XKhucS4?t=2272 https://youtu.be/nlt4XKhucS4?t=2272 in response to a question that you apparently pass PNGs and other binary content "straight through" (in the context of file upload), ie bypassing the WAF. Given things like... - webpage in JPEG (http://lcamtuf.coredump.cx/squirrel/ http://lcamtuf.coredump.cx/squirrel/, https://news.ycombinator.com/item?id=12262470 https://news.ycombinator.com/item?id=12262470, https://news.ycombinator.com/item?id=4209052 https://news.ycombinator.com/item?id=4209052), - JavaScript in EXIF (https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-ex...) - PHP in EXIF (https://web.archive.org/web/20130708132109/https://websec.io/2012/09/05/A-Silent-Threat-PHP-in-EXIF.html https://web.archive.org/web/20130708132109/https://websec.io...) - HTML+JavaScript+1021 byte demo inside PNG: https://news.ycombinator.com/item?id=24824299 https://news.ycombinator.com/item?id=24824299, http://www.p01.org/MONOSPACE/ http://www.p01.org/MONOSPACE/ (general NB: "Packed version" link under "Additional links" actually loads the demo for me in Chrome, but clicking through from HN and loading the URL directly doesn't - some sort of bizarre CORS-related thing?) ...I presume the status quo has changed somewhat here. Hearing how/what's going on in this space would be very interesting.
anotherhue 6y ago
Do you get to go 'off-call' then?
- jgrahamc 6y ago
  I haven't been on call for that code for a long time. There's a whole team that works on it and has been improving it for quite a few years.
  - anotherhue 6y ago
    Glad to hear! It's a little jarring that the Web-UI throws it's hands in the air when you add a few boolean clauses so I look forward to some improvements there. Cloudflare is, in general, a delight to use.
    - jgrahamc 6y ago
      Email me (jgc) any issues.
    - alberth 6y ago
      > "Cloudflare is, in general, a delight to use." Agreed. There's lots of companies in the space that Cloudflare operates in. Cloudflare is the "macOS/Apple" of the market whereas their competition is the unwieldily mess that is "Linux".
      - luckylion 6y ago
        On a technical level: I agree, it works great. Their dashboard needs some work. What I find annoying is mostly the "load everything in single ajax requests, but don't set the size of it in HTML", which makes elements jump around while stuff above them is being rendered. That's very annoying when your connection isn't great for some reason and buttons get replaced by different buttons. I get it, "use the API", but when you need to do something manually, I'd prefer a higher total load time over something that's asking for mistakes. There's a good reason Google is heavily advising to get rid of layout shifts. The other thing I took personally was them removing the "remember me" functionality in a two step process: first it was broken, and then they removed the feature alltogether it instead of fixing.
alberth 6y ago
Sad to see LuaJIT is no longer in use. LuaJIT was so far ahead of it's time (and still is), it's sad to see the current state of it since Mike Pall has moved on. The learned lesson for me is that, even if you have a vastly superior offering - the success of something is hugely correlated to the community behind it.
- fsfod 6y ago
  Mike Pall is still working on LuaJIT just last week he pushed the first stage of string buffer support sponsored by some network company https://github.com/LuaJIT/LuaJIT/commit/4c6b669c419f313306b9e6ee43be4ad5f6d73ec6 https://github.com/LuaJIT/LuaJIT/commit/4c6b669c419f313306b9...
  - exikyut 6y ago
    *Activates [APPLAUSE] indicator*
butt_hugger 6y ago
Do you have any plans to stop ruining the open and secure web?
exikyut 6y ago
Wow. I was imagining the possibilities of being able to share some of the basic techniques behind the system, but being able to have an actual MWE of real racing stripes™ is cooler than I could ever have imagined would be viable. Thanks.

lovedswain 6y ago

Sorry, you don't get to market to folk who're willing to read the words "Magic Quadrant" and post that shit to HN at the same time. Pick your poison! We're a technical crowd, remember? Shudder. I wonder if the CloudFlare folk will be returning to their offices wearing suits when lockdown is over.

ChrisArchitect 6y ago

Recently for a few sites I run I realized I am increasingly verrry dependent or leaning heavily on this Firewall to keep so much Bot and spam traffic off my site.

Farahqq 6y ago

Farajqq

Farahqq 6y ago