5 ms·

Scaling CloudFlare's Massive WAF

68 points by mxpxrocks10 12y ago

puppetmaster3 12y ago
I was always wondering what CloudFare does.
mxpxrocks10 12y ago
Also, I want to point out that many people at Cloudflare were involved with the optimization of the WAF at Cloudflare including @agentzh https://twitter.com/agentzh https://twitter.com/agentzh He also did a fantastic presentation at nginxconf!
- jgrahamc 12y ago
  Yes, agentzh helped a lot once I'd written the initial WAF code.
- meowface 12y ago
  agentzh's work is amazing. Real definition of a 10x or 100x developer.
- mtourne 12y ago
  agentzh has certainly helped CloudFlare a lot by delivering an amazing Lua framework, which CloudFlare's WAF rule are written in. He has also been instrumental in the development of CloudFlare's Core CDN v2, aka cloudflare-nginx [1]. (I wrote a good chunk of said Core CDN v1 and v2, sometime ago). [1] http://blog.cloudflare.com/2013-refactoring-2014-stepping-on-the-gas/ http://blog.cloudflare.com/2013-refactoring-2014-stepping-on...
- deleted 12y ago
  [deleted]
- eugeneionesco 12y ago
  Here's the video from nginxconf https://www.youtube.com/watch?v=Z0fQabvVhIk https://www.youtube.com/watch?v=Z0fQabvVhIk
  - mxpxrocks10 12y ago
    thanks for posting this - woot woot.
seekingtruth 12y ago
How soon until botnets & malware routinely bypass DNS and instead use host files compiled from simple subdomain pings (and other vectors for IP address leaks) and passed about like password lists?
- nacs 12y ago
  If the target server is setup to only accept requests from certain IPs like the Cloudflare IPs then this shouldn't be a problem.
  - seekingtruth 12y ago
    That doesn't seem to be common.
    - mxpxrocks10 12y ago
      it would be if such a list got distributed. Simple IP tables or webserver config. Could you think of a way to make it easier?
stevekemp 12y ago
Cloudflare block comment-spam? That's pretty interesting to hear and not a trivial problem. I've been running http://blogspam.net/ http://blogspam.net/ for the past few years to filter comment-spam from blogs, forums, etc, and it isn't an easy thing to manage.
150 12y ago
I'm sure they are doing a lot of great work. However, I really do not like the idea of having one company to serve all major websites of the internet. Should one not focus on a better solution to ddos-attacks than putting everything into the hands of a single entity..?
- eugeneionesco 12y ago
  >I really do not like the idea of having one company to serve all major websites of the internet So what are you doing about this? Do you have started working on something one can contribute to?
  - 150 12y ago
    Does one need to offer an alternative for voicing a concern?
- medecau 12y ago
  I guess it comes down to how easy it is to set up and forget about it. It may not be what the general user likes but it is so easy to set up that it becomes prevalent. Unlike sibling I won't ask what you've done in this regard but I'll leave the request for a list of what alternatives are currently available.
  - 150 12y ago
    Fully agree that the ease of use combined with a lack of alternatives makes a compelling argument pro CF. And I'm afraid I cannot think of a better solution than at least hope for a forseeable competition will lead to the big sites being spread among several reverse-proxying-companies - instead of them all being served by CF. So far, they seem to be doing great work and consequently outperform everyone else.
- davidy123 12y ago
  It's not hard to make your own Cloudflare-alike. I helped bootstrap this for an organization in the non profit space which now serves dozens of threatened web sites. I even created a monitoring/rotation system that takes care of much of the minute to minute work. The hard part outside state funding is making it profitable / sustainable for real emergencies. But the nature of DDOS is it's largely about fighting fire with fire so basically needs a lot of distributed hosts. This is an area I'd like to see a peer solution be successful, a bittorrent for hosting with no central dependencies.

68 points by mxpxrocks10 12y ago

I was always wondering what CloudFare does.

Also, I want to point out that many people at Cloudflare were involved with the optimization of the WAF at Cloudflare including @agentzh https://twitter.com/agentzh https://twitter.com/agentzh He also did a fantastic presentation at nginxconf!

jgrahamc 12y ago
Yes, agentzh helped a lot once I'd written the initial WAF code.
meowface 12y ago
agentzh's work is amazing. Real definition of a 10x or 100x developer.
mtourne 12y ago
agentzh has certainly helped CloudFlare a lot by delivering an amazing Lua framework, which CloudFlare's WAF rule are written in. He has also been instrumental in the development of CloudFlare's Core CDN v2, aka cloudflare-nginx [1]. (I wrote a good chunk of said Core CDN v1 and v2, sometime ago). [1] http://blog.cloudflare.com/2013-refactoring-2014-stepping-on-the-gas/ http://blog.cloudflare.com/2013-refactoring-2014-stepping-on...
deleted 12y ago
[deleted]
eugeneionesco 12y ago
Here's the video from nginxconf https://www.youtube.com/watch?v=Z0fQabvVhIk https://www.youtube.com/watch?v=Z0fQabvVhIk
- mxpxrocks10 12y ago
  thanks for posting this - woot woot.

seekingtruth 12y ago

How soon until botnets & malware routinely bypass DNS and instead use host files compiled from simple subdomain pings (and other vectors for IP address leaks) and passed about like password lists?

nacs 12y ago
If the target server is setup to only accept requests from certain IPs like the Cloudflare IPs then this shouldn't be a problem.
- seekingtruth 12y ago
  That doesn't seem to be common.
  - mxpxrocks10 12y ago
    it would be if such a list got distributed. Simple IP tables or webserver config. Could you think of a way to make it easier?

stevekemp 12y ago

Cloudflare block comment-spam? That's pretty interesting to hear and not a trivial problem. I've been running http://blogspam.net/ http://blogspam.net/ for the past few years to filter comment-spam from blogs, forums, etc, and it isn't an easy thing to manage.

150 12y ago

I'm sure they are doing a lot of great work. However, I really do not like the idea of having one company to serve all major websites of the internet. Should one not focus on a better solution to ddos-attacks than putting everything into the hands of a single entity..?

eugeneionesco 12y ago
>I really do not like the idea of having one company to serve all major websites of the internet So what are you doing about this? Do you have started working on something one can contribute to?
- 150 12y ago
  Does one need to offer an alternative for voicing a concern?
medecau 12y ago
I guess it comes down to how easy it is to set up and forget about it. It may not be what the general user likes but it is so easy to set up that it becomes prevalent. Unlike sibling I won't ask what you've done in this regard but I'll leave the request for a list of what alternatives are currently available.
- 150 12y ago
  Fully agree that the ease of use combined with a lack of alternatives makes a compelling argument pro CF. And I'm afraid I cannot think of a better solution than at least hope for a forseeable competition will lead to the big sites being spread among several reverse-proxying-companies - instead of them all being served by CF. So far, they seem to be doing great work and consequently outperform everyone else.
davidy123 12y ago
It's not hard to make your own Cloudflare-alike. I helped bootstrap this for an organization in the non profit space which now serves dozens of threatened web sites. I even created a monitoring/rotation system that takes care of much of the minute to minute work. The hard part outside state funding is making it profitable / sustainable for real emergencies. But the nature of DDOS is it's largely about fighting fire with fire so basically needs a lot of distributed hosts. This is an area I'd like to see a peer solution be successful, a bittorrent for hosting with no central dependencies.