4 ms·

I notice that you can have Pushpin talk directly via ZMQ. I would love to see an example ZeroMQ-based Ruby "web server" that can be used a backend for Pushpin

by kondro 12y ago

I notice that you can have Pushpin talk directly via ZMQ. I would love to see an example ZeroMQ-based Ruby "web server" that can be used a backend for Pushpin to avoid the HTTP stack on the application layer.

jkarneges 12y ago
The communication protocol is called "ZHTTP" which works in either REQ/REP or PUSH/SUB modes. It basically wraps HTTP in ZMQ messages. We submitted a draft here: http://rfc.zeromq.org/spec:33 http://rfc.zeromq.org/spec:33 I'm not much of a Ruby developer, but I just committed a Python example using the REQ/REP interface. It should be readable enough so you can get an idea of what's involved: https://github.com/fanout/pushpin/blob/master/tools/zhttpreqhandler.py https://github.com/fanout/pushpin/blob/master/tools/zhttpreq... To activate the realtime stuff, you send the same Grip-* headers as you would with normal HTTP. A word of caution: while this path is more optimized, it is super bare bones. As soon as you start doing anything advanced, you may find yourself wishing you were using a real web framework (I know, because this happened to me once already! haha).
- kondro 12y ago
  Thanks for this, probably enough to get started. The plan was to create a Rack-compatible injection point that uses ZMQ instead of HTTP which could be used with any of the standard Ruby frameworks, rather than getting too bare-bones. I'm not sure whether there are any real wins here though. Although my real use case is to take queries from Pushpin and put them directly into a persistent queue for processing by workers before them pushing the responses directly back at Pushpin so that we can do a zero-downtime deploy, even if the workers are down for up to 55 seconds (with long-polling).
  - rurounijones 12y ago
    https://github.com/rack-amqp https://github.com/rack-amqp 's Jackalope might be of interest to you. It uses AMPQ rather than ZMQ though.
    - kondro 12y ago
      Looks interesting. May be a good starting point, although it's ultimately Pushpin that makes this model interesting to me.

by kondro 12y ago

jkarneges 12y ago

The communication protocol is called "ZHTTP" which works in either REQ/REP or PUSH/SUB modes. It basically wraps HTTP in ZMQ messages. We submitted a draft here: http://rfc.zeromq.org/spec:33 http://rfc.zeromq.org/spec:33 I'm not much of a Ruby developer, but I just committed a Python example using the REQ/REP interface. It should be readable enough so you can get an idea of what's involved: https://github.com/fanout/pushpin/blob/master/tools/zhttpreqhandler.py https://github.com/fanout/pushpin/blob/master/tools/zhttpreq... To activate the realtime stuff, you send the same Grip-* headers as you would with normal HTTP. A word of caution: while this path is more optimized, it is super bare bones. As soon as you start doing anything advanced, you may find yourself wishing you were using a real web framework (I know, because this happened to me once already! haha).

kondro 12y ago
Thanks for this, probably enough to get started. The plan was to create a Rack-compatible injection point that uses ZMQ instead of HTTP which could be used with any of the standard Ruby frameworks, rather than getting too bare-bones. I'm not sure whether there are any real wins here though. Although my real use case is to take queries from Pushpin and put them directly into a persistent queue for processing by workers before them pushing the responses directly back at Pushpin so that we can do a zero-downtime deploy, even if the workers are down for up to 55 seconds (with long-polling).
- rurounijones 12y ago
  https://github.com/rack-amqp https://github.com/rack-amqp 's Jackalope might be of interest to you. It uses AMPQ rather than ZMQ though.
  - kondro 12y ago
    Looks interesting. May be a good starting point, although it's ultimately Pushpin that makes this model interesting to me.