5 ms·
Nginx 0mq transport layer for communicating with upstream servers
- njharman 15y agoWhy? Is it faster, more robust, something, anything????
- piotrSikora 15y agoProof-of-concept, nothing else at this point.
- alberth 15y agoHow is this different than Mongrel2? Wasn't Mongrel2 created solely because NGINX didn't support ZeroMQ. @ZedShaw, thoughts - comments?
- zrail 15y agoFrom what I understand, core Nginx decided that supporting ZeroMQ was a non-starter because of the threading model that ZeroMQ uses. I'm not sure if that's just not true anymore, or if the restrictions that this new module places on it (namely, exclusively REQ/REP pairs) make that not apply.
- piotrSikora 15y ago> if the restrictions that this new module places on it (namely, exclusively REQ/REP pairs) make that not apply. ZeroMQ I/O threads are still being used with REQ/REP, so this doesn't change anything in that matter. Also, exclusive use of REQ/REP pair isn't restriction, but current limitation of the module ;)
- swah 15y agoI'm not sure how well adopted mongrel2 has been. There ain't many commits by Zed these days and I don't see blog posts about using it. I don't know why though.
- alberth 15y agoCommits to Mongrel2 are happening near daily. It's in the development branch. https://github.com/zedshaw/mongrel2/commits/develop https://github.com/zedshaw/mongrel2/commits/develop
- daltonmatos 15y agoI've been using mongrel2 pretty much since the beginning. I host my own websites with it. Although they are very low traffic I ran some performance tests using blitz.io and the results were impressive. Less than 1% conn errors with a 512MB linode and a simple Django app (no database). I'm also blogging about mongrel2 lately: http://daltonmatos.wordpress.com http://daltonmatos.wordpress.com
- alexgartrell 15y agoI was one of the first committers on Mongrel2 (actually, second behind Zed) and I still am (though mostly as a historical accident, I haven't contributed anything of substance since SSL). In my opinion, this is great. I think Mongrel2 is an excellent proof-of-concept, and if it provided the impetus for a much more widely used project like nginx to go down a similar path (though it's just a module right now), then everyone wins. That said, this certainly isn't in nginx mainline, and it's very feature poor as compared to Mongrel2. One could not simply swap in nginx with this module for an existing Mongrel2 installation.
- igrigorik 15y agoAlex, do you know if Mongrel2 have any interesting production deployments? In terms of visibility, throughput, or any other metrics. Outside from the architecture discussions, I haven't see any mentions of it.. (and I'm a fan of the architecture).
- piotrSikora 15y ago> it's very feature poor as compared to Mongrel2. One could not simply swap in nginx with this module for an existing Mongrel2 installation. That's because this is _transport_module_ and what Zed did with Mongrel2 is build a level 7 protocol on top of ZeroMQ messages. Even if this would support PUSH/PULL + PUB/SUB sockets right now, you would still need an _upstream_module_ that understands Mongrel2's level 7 protocol.
- piotrSikora 15y agoHead's up, I've started working on both: - PUSH/PULL + PUB/SUB support for ngx_zeromq (transport module), - ngx_mongrel2 (upstream module). Should be ready sometime next week.
- technoweenie 15y agoMongrel2's ZMQ protocol was designed for lightweight messages against async handlers. Instead of a synchronous ZMQ REQ/REP pair that this nginx module uses, Mongrel2 used a combination PUSH/PULL sockets to send the request, and PUB/SUB to get a response. Say you want to map /chat to a handler. Mongrel2 opens a PUSH socket, and a SUB socket connecting to one or more chat handlers. The handlers open a PULL and PUB socket. Mongrel2 will spread requests to /chat across the available handlers, and receive all published responses. It's a pretty smart way to handle multiple handlers receiving requests at the same time. It's been a while since I looked at Mongrel2, so it's very possible I have some of the details wrong...
- piotrSikora 15y ago> How is this different than Mongrel2? It has nothing in common with Mongrel2 other than the fact that both use ZeroMQ transport instead of TCP.
- andrewvc 15y agoCool stuff, but I'm wondering why they went with REP/REQ instead of ROUTER/DEALER. That would dramatically reduce the number of file descriptors needed (indeed, a single socket could handle any number of backends / requests). Also, the fact that ZMQ is message, not stream, oriented makes it non-ideal here. You get the whole response in one big chunk. Across an internal network, this may not be a huge deal, but it's not a desirable property. It's conceivable that a variant that broke responses up into discrete messages might be better, but this would have more complication in implementation, and I wonder how much it'd have to offer over straight HTTP. Also, I wonder if the true ideal protocol for this is SPDY without SSL. It just seems like a no-brainer.
- themcgruff 15y agoThis is just an initial spike. The conversation started here: https://github.com/chaoslawful/lua-nginx-module/issues/35 https://github.com/chaoslawful/lua-nginx-module/issues/35.
- JoshTriplett 15y ago> Also, I wonder if the true ideal protocol for this is SPDY without SSL. SPDY does not exist without SSL, nor will it. Also, SSL does not add anywhere near enough overhead to matter for this kind of application, especially when you just leave the connection established. SPDY would work quite well as a protocol for this purpose.
- andrewvc 15y agoSpeed aside, maintaining certs for all your backend servers would suck.
- re 15y ago> SPDY does not exist without SSL, nor will it. Can you elaborate on this, or provide a link with more context? Almost all TLS/NPN references are missing from the current SPDY draft spec. (I've found webpages that repeat the SSL requirement assertion, but none that source it or explain why.)
- splitrocket 15y agoI've used frickle's ngx_cache_purge in production for over a year: it is rock solid. I'm looking forward to finding a use for this plugin! thanks!