4 ms·

This kind of validates an idea I've been flirting with: a Heroku-like service which routes requests via AMQP or similar message broker and actually exposes the

This kind of validates an idea I've been flirting with: a Heroku-like service which routes requests via AMQP or similar message broker and actually exposes the routing dynamics to the client apps. From a naive, inexperienced view the idea of having web nodes "pull" requests from a central queue rather than the queue taking uneducated guesses seems to be a no-brainer. I can see this making long-running requests (keep-alive, streaming, etc) a bit more difficult, but not impossible. What am I missing? This seems so glaringly obvious that it must have been done before...

bdittmer 14y ago
I believe Mongrel2 (http://mongrel2.org http://mongrel2.org) is close to what you're talking about. It uses zeromq (http://www.zeromq.org http://www.zeromq.org) to talk to a backend application and the backend application talks back to the web server using the same protocol. Not exactly nodes pulling messages off a queue, but closer to something like that?
dblock 14y ago
The pull model is very hard to implement because the router behaves like a proxy for a much larger set of dynos (think tens of thousands). When you have 10K clients yielding "i'm available" 10 times a second, you have a nightmare, it's not sustainable. A possible solution for the proxy and the dynos to agree on a protocol where the proxy passes a request to the dyno and the latter can give up with a status code that says "retry with another dyno". This could go on to up to the 30s timeout limit that Heroku has now.
- jacques_chester 14y ago
  In Mongrel2, app servers subscribe to a named ZeroMQ queue and, when they're done, they send the response on a different queue. You can actually configure an arbitrary number of different queues if you like, switching on request path and some other stuff I don't just now recall.
X-Istence 14y ago
That is exactly how an app I develop at $WORK works. Requests come in on a front-end, it gets passed off to a router that has multiple different workers connected. The workers send a request to the router letting the router know that they are ready to start responding to requests. The router hands the worker a request, marks the request as being worked on, and moves on to the next request. It is a basic Least Recently Used queue at that point, and if all workers are busy but worker number 3 which received work last finished first, he gets handed new work instantly. The worker then sends the request back to the router, which sends it back to the appropriate front-end that originally responded to the user. We are using ZeroMQ for our communication. For our use case we can handle around 200 requests a second from a TCP/IP connected client to our router, to a worker and back to a client. That is with 3 backend workers, which are hitting the disk/database. It has worked very well for us.
themgt 14y ago
The problem with this model is you'd need to re-engineer all existing web applications to work with the new service. If I just want to boot up an open-source Rails/Django/node.js/etc app on your service, how would I do it?
jacques_chester 14y ago
It has: https://github.com/paulj/trapeze https://github.com/paulj/trapeze You might also be interested in what's been done in the literature; look under "Staged Event Driven Architecture" aka SEDA. Mongrel2 and OK Web Server both support the SEDA approach.

bdittmer 14y ago

I believe Mongrel2 (http://mongrel2.org http://mongrel2.org) is close to what you're talking about. It uses zeromq (http://www.zeromq.org http://www.zeromq.org) to talk to a backend application and the backend application talks back to the web server using the same protocol. Not exactly nodes pulling messages off a queue, but closer to something like that?

dblock 14y ago

The pull model is very hard to implement because the router behaves like a proxy for a much larger set of dynos (think tens of thousands). When you have 10K clients yielding "i'm available" 10 times a second, you have a nightmare, it's not sustainable. A possible solution for the proxy and the dynos to agree on a protocol where the proxy passes a request to the dyno and the latter can give up with a status code that says "retry with another dyno". This could go on to up to the 30s timeout limit that Heroku has now.

jacques_chester 14y ago
In Mongrel2, app servers subscribe to a named ZeroMQ queue and, when they're done, they send the response on a different queue. You can actually configure an arbitrary number of different queues if you like, switching on request path and some other stuff I don't just now recall.

X-Istence 14y ago

That is exactly how an app I develop at $WORK works. Requests come in on a front-end, it gets passed off to a router that has multiple different workers connected. The workers send a request to the router letting the router know that they are ready to start responding to requests. The router hands the worker a request, marks the request as being worked on, and moves on to the next request. It is a basic Least Recently Used queue at that point, and if all workers are busy but worker number 3 which received work last finished first, he gets handed new work instantly. The worker then sends the request back to the router, which sends it back to the appropriate front-end that originally responded to the user. We are using ZeroMQ for our communication. For our use case we can handle around 200 requests a second from a TCP/IP connected client to our router, to a worker and back to a client. That is with 3 backend workers, which are hitting the disk/database. It has worked very well for us.

themgt 14y ago

The problem with this model is you'd need to re-engineer all existing web applications to work with the new service. If I just want to boot up an open-source Rails/Django/node.js/etc app on your service, how would I do it?

jacques_chester 14y ago

It has: https://github.com/paulj/trapeze https://github.com/paulj/trapeze You might also be interested in what's been done in the literature; look under "Staged Event Driven Architecture" aka SEDA. Mongrel2 and OK Web Server both support the SEDA approach.