7 ms·
WebSocket proxy support added to nginx trunk
- thruflo 14y agoThat's excellent news! Having explored the workarounds, involving additional layers of deployment stack and / or manual compilation, I'll be delighted when this gets released and package managed.
- Hello71 14y ago*packaged
- deleted 14y ago[deleted]
- gritzko 14y agoAt last!
- andyfleming 14y agoAgreed! I'm excited to see this functionality making its way into nginx. Any details on when it will be officially released and make its way into repositories?
- alexjarvis 14y agoThis is nice to see. I wonder how it compares to HAProxy http://haproxy.1wt.eu/ http://haproxy.1wt.eu/ which also works with SSL and Secure WebSockets (wss://).
- jlongster 14y agoHow long does it usually take for things to go from trunk to release? This will be awesome when it's ready because I can keep my production stack really simple. All the existing solution for proxying sockets seemed way too complex, and I typically just run them on different IPs, but even that's annoying.
- moonboots 14y agoThe next dev release with websocket proxy support is scheduled for tomorrow [1]. [1] http://trac.nginx.org/nginx/roadmap http://trac.nginx.org/nginx/roadmap
- andyfleming 14y agoAny date/guess on a production release?
- moonboots 14y agoNginx's development branch is generally considered production ready [1]. I trust the nginx core team not to introduce potentially problematic changes. As an example, the recent spdy integration [2] has lived in a parallel branch for over half a year. The dev branch is officially upgraded to stable infrequently, maybe once a year. [1] http://blog.bigdinosaur.org/nginx-dev-or-stable/ http://blog.bigdinosaur.org/nginx-dev-or-stable/ [2] http://nginx.org/patches/spdy/ http://nginx.org/patches/spdy/
- jlongster 14y agoThanks, that's really helpful! I wouldn't have considered that. It still seems risky using a dev branch for my main web server, but I'll look into it.
- alptrv 14y agoCan anyone explain how proxying websockets wouldn't hit the port number limit on the proxy server? If I understand correctly, when we would proxy tcp connections the workers on the backend server would able to hold only up to 64k connections to the proxy server (in the case when proxy server has only one IP-address)? Am I missing something?
- wmf 14y agoYep, you need multiple IP addresses and other tricks to create over 64K connections to a single backend: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-3 http://www.metabrew.com/article/a-million-user-comet-applica... One workaround that comes to mind is to have each backend listen on multiple ports.
- alexkus 14y agoYou don't need any special tricks in the backend. The only limits on the backend are the number of file descriptors. You only need special tricks when you want to simulate a large number of client connections from a single machine (or bunch of machines). The client(s) have a requirement of a unique IP/source-port combination; not the server.
- hahainternet 14y agoThanks for correcting this. People misunderstand how tcp connections work.
- wmf 14y agoBut that case is exactly what we're talking about: a single nginx making over 64K connections to a single backend server. You either need nginx to have multiple IPs or the backend to have multiple listening ports.
- alptrv 14y agoBut I assumed that backend isn't connected to the client directly, it's connected to the proxy server, and the proxy server holds tcp connections to the clients, and of course can hold millions of them.
- chatmasta 14y agoCould someone explain why this is helpful? What's an example of a use case where this becomes one of the better solutions?
- craigching 14y agoSorry if this is obvious, but you need it when nginx is your front-end and you want to use web sockets without exposing your backend. A lot of people deploy nginx in their DMZ because of its security and don't want to expose their backend servers, e.g. JEE or Ruby or what have you.
- Wintamute 14y agoAFAIK there aren't that many robust approaches for reverse proxying web sockets. Apache mod_proxy doesn't work with web sockets, nginx (until now) only worked with websockets if you compiled and patched an unstable dev branch yourself, and not everyone was happy to get their hands dirty with HAProxy. An alternative was to build your own reverse proxy in Node.JS using node-http-proxy, but again, not everyone's cup of tea. Use cases of websockets enabled reverse proxies are for when you have a number of websockets based apps all sitting on the same box or in the same private local network and you need a way to route incoming traffic to them based on the hostname or path or whatever.
- imsofuture 14y agoFWIW, I've found HAProxy even more joyous to work with than nginx.
- exratione 14y agoSee this series of posts: http://www.exratione.com/2012/07/proxying-websocket-traffic-for-nodejs-the-present-state-of-play/ http://www.exratione.com/2012/07/proxying-websocket-traffic-... http://www.exratione.com/2012/08/websockets-over-ssl-stunnel-varnish-nginx-nodejs/ http://www.exratione.com/2012/08/websockets-over-ssl-stunnel... http://www.exratione.com/2012/12/websockets-over-ssl-haproxy-nodejs-nginx/ http://www.exratione.com/2012/12/websockets-over-ssl-haproxy... "So let us say that you are developing a Node.js / Socket.IO application that both uses websockets and serves files the old-fashioned way, such as through Express - this is a fairly common situation. You want to have an HTTP proxy server rather than Node.js field all incoming traffic, so that you can set up load balancing, route requests for static files to some other server process, avoid having to set up SSL configuration in Node.js, and so forth. "Unfortunately at this point in time it isn't completely straightforward to proxy both websocket traffic and ordinary web requests for Node.js through a single port - which would be the ideal situation. It becomes even less straightforward if you want to use SSL. The following is a brief overview of present options as of early Q3 2012." "The up-front summary: if you are reading this after Nginx adds support for proxying websocket traffic to Node.js (supposedly coming up in version 1.3), then everything is rainbows and unicorns - just use Nginx. If you are reading this prior to Nginx support for proxying of websocket traffic, then you will likely have to do more work and investigation in order to create a good proxy setup for your servers."
- tlrobinson 14y agoDoes this support sticky sessions? Normally I think it's best for HTTP backends to be stateless, but WebSockets open up interesting possibilities for RPC solutions like NowJS (https://github.com/Flotype/now https://github.com/Flotype/now), dnode (https://github.com/substack/dnode https://github.com/substack/dnode), and others (https://github.com/kriskowal/q-connection https://github.com/kriskowal/q-connection) that benefit from keeping state in memory in the app servers. I'd be interested to hear about other load balancing techniques for WebSockets.
- runesoerensen 14y agoI'd expect the ip_hash directive to work as usual http://wiki.nginx.org/HttpUpstreamModule#ip_hash http://wiki.nginx.org/HttpUpstreamModule#ip_hash although it's not exactly sticky sessions. There's a separate module for that https://github.com/michaelneale/nginx-sticky-module https://github.com/michaelneale/nginx-sticky-module
- mattyb 14y agoHAProxy can, reportedly, load balance WebSocket connections: http://blog.exceliance.fr/2012/11/07/websockets-load-balancing-with-haproxy/ http://blog.exceliance.fr/2012/11/07/websockets-load-balanci... dotCloud's Hipache also looks like it was basically written to solve this problem: https://github.com/dotcloud/hipache https://github.com/dotcloud/hipache
- jabr 14y agoWebSockets are persistent connections, not a state-management trick spanning multiple connections/requests. The only time nginx is involved in "routing" the socket is on the initial connection. Once established, nginx is simply relaying data over an active, persistent TCP/unix sockets. For the initial connection, you can use any of nginx's normal proxy load balancing options (ip_hash, least_conn, etc).
- pixie_ 14y agoThis should fix my problem of not being able to connect to my site though a corporate firewall because I run multiple tornado instances on a single server on different ports. Before nginx wasn't able to route port 80 requests to the chat sub-domains, but now it will be able to.. w00t
- adamkittelson 14y agoRight on! I'm using nginx and thin, but I'm currently having to use varnish for proxying my websocket connections to thin because nginx can't do it. Then I have to use stunnel for ssl termination because varnish doesn't support it. I'll be able to cut both of those pieces of software out now.
- densone 14y agoWhy not just use haproxy?
- mattdawson 14y agoIt's been over a year since I had to set this up, but I used HAProxy in a similar setup and still needed stunnel. In the whole of my DevOps career, getting SSL working with web sockets was one of the most frustrating.
- imsofuture 14y agoHAProxy dev releases have had SSL support for the past 3-4 months. http://blog.jeffzellner.com/work/2013/01/25/websockets--ssl-via-haproxy/ http://blog.jeffzellner.com/work/2013/01/25/websockets--ssl-...
- densone 14y agoWe are running 1.5 latest dev compiled with ssl here @basho. So far so good. https://argv0.data.riakcs.net:8443/bailis_gosling.jpg https://argv0.data.riakcs.net:8443/bailis_gosling.jpg
- theturtle32 14y agoI second the success reports of HAProxy 1.5dev with SSL support. It's been stable for me over the last month, and more performant than using STunnel for SSL, which is what I'd been using for the last 1.5 years with Websockets. HAProxy is a remarkable, wicked fast, and very stable product.
- jenandre 14y agoWe're using this to reverse proxy websockets over SSL and it's simple and wonderfully stable. https://github.com/nodejitsu/node-http-proxy https://github.com/nodejitsu/node-http-proxy
- donpdonp 14y agodo you know how this compares to bouncy? I use bouncy as my web proxy including websockets and SNI. I love the flexability of writing the proxy in javascript. https://github.com/substack/bouncy https://github.com/substack/bouncy
- jenandre 14y agoIt looks comparable to me... I hadn't seen bouncy before though, so I'll have to check it out if I run into any problems with node-http-proxy. I agree, having the power to write your proxy code in javascript is really nice, especially compared to having to struggle with getting some infrastructure's DSL to work for you (e.g. nginx, varnish's).
- kevinfat 14y agoAre WebSocket connections persistent to a single backend? That is, suppose Nginx is in front of two Unicorn processes. Will Nginx maintain a mapping table of client browser connection to Unicorn process, so that messages will always hit the same Unicorn process? Does that mean the Unicorn processes don't behave in a "stateless" manner? Then what happens to the websocket if a Unicorn process dies and is restarted? What does the client browser do? What does Nginx do? What happens to the websocket connections if Nginx dies and there is a failover to a backup Nginx instance?
- jabr 14y agoAll this change in nginx does is allow WebSocket connections to be proxied, so your questions don't really make sense. WebSocket's work like TCP sockets, and your backend server still needs to manage the socket directly. For example, Unicorn is not a WebSocket server, though there are WebSocket-capable servers derived from Unicorn (e.g. http://rainbows.rubyforge.org/ http://rainbows.rubyforge.org/). If your backend dies or drops the connection, the WebSocket closes, and the client will have to reconnect. If it was just dropped, you can use nginx's ip_hash proxy option to send the reconnect to the same backend server, but it would be a new WebSocket, and the client/server will have to recreate the session state.
- cft 14y agoI find it quite interesting that Android default browser (and thus WebView) is the only modern browser that still does not support WebSockets. I wish Google shed some light on why this is. Unupdatable Android browser is the modern IE6.
- wmf 14y agoIsn't Browser essentially unmaintained in favor of Chrome (which itself is somewhat crippled compared to the desktop version)?
- cft 14y ago1. Even Android 4+ devices that support Chrome sold in major carrier stores still have Android browser as the default (which means that most users will use it) 2. Even in Android 4.2 WebView is based on default browser, meaning that any hybrid Android app does not have WebSockets built in. This is the main reason why WebSockets are not ready for production.
- mxxx 14y agoeven with proper browser support, websockets are fairly flaky. there's all kinds of issues with firewalls/proxies that can present problems. i've been working on an app recently that was (hoping) to implement websockets, but at this stage it seems xhr-polling is still the best bet...
- igrigorik 14y agoOne word: SSL. Works well, and reliably with websockets.
- SideburnsOfDoom 14y ago> (hoping) to implement websockets, but at this stage it seems xhr-polling is still the best bet.. There are libraries like SignalR that will try websockets, but fall back to long polling if the client does not suport it.
- pornel 14y agoOr if you don't need realtime upload (only realtime download) then use Server-Sent Events, which is HTTP-compatible and thus supported by proxies already.
- chrislea 14y agoHi all. I make Ubuntu packages of the Nginx development releases, so if you want to try out the WebSocket proxying and you're using Ubuntu, these should make it pretty easy to do: https://launchpad.net/~chris-lea/+archive/nginx-devel https://launchpad.net/~chris-lea/+archive/nginx-devel I've written up a very quick tutorial on how to set things up here: https://chrislea.com/2013/02/23/proxying-websockets-with-nginx/ https://chrislea.com/2013/02/23/proxying-websockets-with-ngi... Hope people find these useful.