10 ms·

Isnt't websocket used mainly for event based communication to clients ? That would seem like something absolutely impossible to cache or proxy, not because of t

by bsaul 12y ago

Isnt't websocket used mainly for event based communication to clients ? That would seem like something absolutely impossible to cache or proxy, not because of the technology itself but because the content seems almost specific every time. What would the be the business case for using cloudflare with websockets ?

jgrahamc 12y ago
People want to protect all their public 'web' IPs using CloudFlare. That means they have to be able to protect anything that uses WebSockets, so we have to proxy it. Now we do.
- TazeTSchnitzel 12y ago
  Right. I had a site using CloudFlare and it was inconvenient having to have a bypass rule for a subdomain (ws.example.com) for WebSocket and having everything else on the main domain.
- bsaul 12y ago
  Thanks, i completely forgot about the dos aspect.
dugmartin 12y ago
As I understand it Cloudflare doesn't cache non-asset traffic (like html pages) by default unless you put it on a whitelist on a paid plan. I think the business case is to protect against DDOSes by having Cloudflare route the traffic off your origin servers.
- xxdesmus 12y ago
  You can setup a PageRule (basically an exception) on any plan level and tell us to cache an entire page (including HTML) if you use the "cache everything" option. This is available at the free level also. With the paid level plans you get more PageRules, but at the free level you still get 3 PageRules. With "cache everything" used correctly (to ensure you're not accidentally telling us to cache something like a login form CloudFlare can easily handle 98%+ of all requests for your high traffic pages.
  - dugmartin 12y ago
    Thanks. I should have looked at your pricing/features page before I commented. I was looking there last week and only 1/2 remembered correctly.
    - xxdesmus 12y ago
      No worries. Just wanted to mention. :) PageRules are extremely powerful. Using "cache everything" can also make a huge difference is setup correctly.
ceejayoz 12y ago
Uncacheable content can still benefit from a CDN, by connecting to a local edge node and having the rest of the traffic go over the CDN's internal network. http://www.cloudflare.com/railgun http://www.cloudflare.com/railgun
andrewstuart2 12y ago
There could be a huge benefit for someone who wants to send websocket frames to say, 100,000 clients. You can't do that with a single server; you'd run out of ports (reuse aside). You'd also have a lot of load on one server. At the cost of a little latency to the true event, you can increase performance and scalability dramatically by sending the event to just a few intermediaries that then broadcast the event to a subset of clients.
- tobz 12y ago
  I don't think this feature functions as a fanout mechanism.
- jkarneges 12y ago
  FWIW, we provide this at Fanout.io. Our service is able to act as a passthrough proxy for WebSockets, not as a DDoS protection service like CloudFlare, but to handle large one->many deliveries as you describe. You can publish a message once, and it can be injected downstream into any number of open connections.
- toast0 12y ago
  > There could be a huge benefit for someone who wants to send websocket frames to say, 100,000 clients. You can't do that with a single server; you'd run out of ports (reuse aside). You would not run out of ports, as long as your clients have some IP diversity. Connections must be unique with the key being the 5-tuple: {Protocol, SourceAddr, SourcePort, DestAddr, DestPort}. If you have one server listening on one port, each client address can theoretically have 65535 connections (in practice, you're unlikely to find a client that will allocate their ports well enough to use all the ports). Assuming your websockets are mostly idle, handling 100k connections for one server doesn't really need that big of a server either, see WhatsApp blog post about 2.2 million connections on one server in 2012: http://blog.whatsapp.com/196/1-million-is-so-2011 http://blog.whatsapp.com/196/1-million-is-so-2011 Caveats: dual 6-core westmere cpu, 100 G ram, normal connection count is lower per server; connections are mostly idle; some tuning required. See also 2.8 M connections documented on page 16 of Erlang Factory presentation: http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf http://www.erlang-factory.com/upload/presentations/558/efsf2...
tobz 12y ago
To compound on everything said here: traditionally, you would either have a set of load balancers, or set of web servers, taking requests directly from clients. If someone wanted to DDoS you, they hit the public IPs of these machines. It can be nearly impossible to mitigate a large enough attack to the point of restoring normal service. You'd most likely be forced to switch to entirely new IPs for a long enough attack event. CloudFlare, in general, accomplishes their DDoS protection by having a super large, geographically redundant network that absorbs the attack before passing legitimate traffic on to your servers. This problem isn't strictly related to HTTP requests - it's really anything running on your servers. If the request is able to get past firewalls, switches, routers, to your machine, it has a chance to eat up resources, and ultimately take down that server / service. WebSockets are being used more and more for event-based communications, as you mentioned, providing real-time updates instead of relying on polling, or long polling hacks. Thus, they're just as viable of an attack vector as directing an attack at port 80. This is where the intermediate layer, provided by CloudFlare, comes into play. The caching aspect of CloudFlare is an important, but separate aspect of their service.
- jgrahamc 12y ago
  CloudFlare, in general, accomplishes their DDoS protection by having a super large, geographically redundant network that absorbs the attack before passing legitimate traffic on to your servers. That's part of the answer. But also we do very high packet rate filtering using stuff like this: http://blog.cloudflare.com/bpf-the-forgotten-bytecode http://blog.cloudflare.com/bpf-the-forgotten-bytecode
  - tobz 12y ago
    I simply opted to call out your network because that is the mechanism that hides the origins, despite the other layers of protection provided by software running at the edge. :)

by bsaul 12y ago

jgrahamc 12y ago

People want to protect all their public 'web' IPs using CloudFlare. That means they have to be able to protect anything that uses WebSockets, so we have to proxy it. Now we do.

TazeTSchnitzel 12y ago
Right. I had a site using CloudFlare and it was inconvenient having to have a bypass rule for a subdomain (ws.example.com) for WebSocket and having everything else on the main domain.
bsaul 12y ago
Thanks, i completely forgot about the dos aspect.

dugmartin 12y ago

As I understand it Cloudflare doesn't cache non-asset traffic (like html pages) by default unless you put it on a whitelist on a paid plan. I think the business case is to protect against DDOSes by having Cloudflare route the traffic off your origin servers.

xxdesmus 12y ago
You can setup a PageRule (basically an exception) on any plan level and tell us to cache an entire page (including HTML) if you use the "cache everything" option. This is available at the free level also. With the paid level plans you get more PageRules, but at the free level you still get 3 PageRules. With "cache everything" used correctly (to ensure you're not accidentally telling us to cache something like a login form CloudFlare can easily handle 98%+ of all requests for your high traffic pages.
- dugmartin 12y ago
  Thanks. I should have looked at your pricing/features page before I commented. I was looking there last week and only 1/2 remembered correctly.
  - xxdesmus 12y ago
    No worries. Just wanted to mention. :) PageRules are extremely powerful. Using "cache everything" can also make a huge difference is setup correctly.

ceejayoz 12y ago

Uncacheable content can still benefit from a CDN, by connecting to a local edge node and having the rest of the traffic go over the CDN's internal network. http://www.cloudflare.com/railgun http://www.cloudflare.com/railgun

andrewstuart2 12y ago

There could be a huge benefit for someone who wants to send websocket frames to say, 100,000 clients. You can't do that with a single server; you'd run out of ports (reuse aside). You'd also have a lot of load on one server. At the cost of a little latency to the true event, you can increase performance and scalability dramatically by sending the event to just a few intermediaries that then broadcast the event to a subset of clients.

tobz 12y ago
I don't think this feature functions as a fanout mechanism.
jkarneges 12y ago
FWIW, we provide this at Fanout.io. Our service is able to act as a passthrough proxy for WebSockets, not as a DDoS protection service like CloudFlare, but to handle large one->many deliveries as you describe. You can publish a message once, and it can be injected downstream into any number of open connections.
toast0 12y ago
> There could be a huge benefit for someone who wants to send websocket frames to say, 100,000 clients. You can't do that with a single server; you'd run out of ports (reuse aside). You would not run out of ports, as long as your clients have some IP diversity. Connections must be unique with the key being the 5-tuple: {Protocol, SourceAddr, SourcePort, DestAddr, DestPort}. If you have one server listening on one port, each client address can theoretically have 65535 connections (in practice, you're unlikely to find a client that will allocate their ports well enough to use all the ports). Assuming your websockets are mostly idle, handling 100k connections for one server doesn't really need that big of a server either, see WhatsApp blog post about 2.2 million connections on one server in 2012: http://blog.whatsapp.com/196/1-million-is-so-2011 http://blog.whatsapp.com/196/1-million-is-so-2011 Caveats: dual 6-core westmere cpu, 100 G ram, normal connection count is lower per server; connections are mostly idle; some tuning required. See also 2.8 M connections documented on page 16 of Erlang Factory presentation: http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf http://www.erlang-factory.com/upload/presentations/558/efsf2...

tobz 12y ago

To compound on everything said here: traditionally, you would either have a set of load balancers, or set of web servers, taking requests directly from clients. If someone wanted to DDoS you, they hit the public IPs of these machines. It can be nearly impossible to mitigate a large enough attack to the point of restoring normal service. You'd most likely be forced to switch to entirely new IPs for a long enough attack event. CloudFlare, in general, accomplishes their DDoS protection by having a super large, geographically redundant network that absorbs the attack before passing legitimate traffic on to your servers. This problem isn't strictly related to HTTP requests - it's really anything running on your servers. If the request is able to get past firewalls, switches, routers, to your machine, it has a chance to eat up resources, and ultimately take down that server / service. WebSockets are being used more and more for event-based communications, as you mentioned, providing real-time updates instead of relying on polling, or long polling hacks. Thus, they're just as viable of an attack vector as directing an attack at port 80. This is where the intermediate layer, provided by CloudFlare, comes into play. The caching aspect of CloudFlare is an important, but separate aspect of their service.

jgrahamc 12y ago
CloudFlare, in general, accomplishes their DDoS protection by having a super large, geographically redundant network that absorbs the attack before passing legitimate traffic on to your servers. That's part of the answer. But also we do very high packet rate filtering using stuff like this: http://blog.cloudflare.com/bpf-the-forgotten-bytecode http://blog.cloudflare.com/bpf-the-forgotten-bytecode
- tobz 12y ago
  I simply opted to call out your network because that is the mechanism that hides the origins, despite the other layers of protection provided by software running at the edge. :)