4 ms·

> If you want to connect to multiple remote servers for high availability purposes, you insert something like HAProxy in the middle. On our PaaS[1] we are runn

by Loic 15y ago

> If you want to connect to multiple remote servers for high availability purposes, you insert something like HAProxy in the middle. On our PaaS[1] we are running ZMQ everywhere and you do not need HAProxy in the middle to get high availability, you do it directly with the right ZMQ devices depending on your requirements. HAProxy is another piece of infrastructure to maintain where you can get HA with the majordomo pattern using several brokers and retrying the requests etc. Check the ZMQ Guide[2], you have nearly everything nicely explained there. So this comment just ring a "warning" for me, the system looks really interesting, but are the ZMQ primitives well enough understood? [1]: http://notes.ceondo.com/mongrel2-zmq-paas/ http://notes.ceondo.com/mongrel2-zmq-paas/ [2]: http://zguide.zeromq.org/page:all http://zguide.zeromq.org/page:all Update: Missing some parts of the comment, stupid me.

bombela 15y ago
Hi, maintainer of zerorpc-python here. If you want to have to open a dozen of different ZMQ sockets to do HA, do like in the guide. With zerorpc-python, we want to use only the messaging performances of ZMQ and multiplex everything on one single socket (streaming, subrpc!, heartbeat and counting). Note that ZMQ have also different behavior depending of which part of the communication is binding/connecting. On a cloud environment, ports are a precious resource.
- Loic 15y ago
  Thanks for the comment. If you use the right devices, you do not really need "a dozen" more sockets for HA. But your justification why you prefer HAProxy is what really matters. Every technology is applied in a given context and your context — cloud with a finite number of ports you need not to waste — makes everything clear. Note that I was contacted a week ago to comment on this project and my comment was basically interesting and looks good but this HAProxy thing does not ring ok. So really, add the context in your readme, you will clear a lot of confusion for people used to ZMQ.
  - bombela 15y ago
    Added to the already long todo ;) Thanks for the feedback. I also believe that it is not possible to build some sort of connected stream between a DEALER and a ROUTER socket if the DEALER round-robing behind your back. ZMQ 3.0 was giving the tool to use a DEALER in a more router fashioned way, sadly, it got removed in 3.1...
  - beagle3 15y ago
    > If you use the right devices, you do not really need "a dozen" more sockets for HA. Would you be able to post a short write up about how to achieve that within the ZeroMQ framework?
    - Loic 15y ago
      Why not, I added it on my list of things to write about. Thank you.
shykes 15y ago
Hi Loic, thanks for the note and it's cool to follow your work from afar. It looks like we get excited about the same stuff :) Disclaimer: I co-founded dotCloud and have the distinctive honor of being the least knowledgeable about zeromq in the entire team. Your comment might have been true 18 months ago - when we first started using zerorpc in production at dotCloud. Since then, we have deployed and scaled hundreds of thousands of applications and I shudder just to imagine how many billions of zeromq messages we have emitted and processed. Believe me we have been through the zeromq guide many times and have experimented with - and abused - many patterns (including majordomo which, as you fail to mention, is not supported out of the box by zeromq and requires a fair amount of custom code of its own). I'm sure zerorpc has many flaws and I know the team looks forward to many constructive debates and - hopefully - patches. But lack of understanding of the zeromq fundamentals, or lack of real-world usage, are 2 things you definitely don't need to worry about :)
- Loic 15y ago
  Hi Solomon, thank you for your nice comments. Bombela very nicely explained the whys HAProxy, with his explanations everything fall in place nicely (and I must say, I will test drive HAProxy with ZMQ).
jpetazzo 15y ago
It's a deliberate trade-off. ØMQ itself doesn't handle HA automatically. As a client, you can create a REQ socket and connect it to multiple redundant devices. But if a device goes down, you will have to handle timeouts and repeat requests yourself. Additionally, ØMQ will not "kick out" the dead endpoint, so you will end up with 1 out of N requests timing out. So you have to add a decent amount of code in your RPC layer to make everything work properly. And you need to run ZMQ devices anyway. Running a local HAProxy with a 5 lines configuration is, IMHO, simpler than running an extra device somewhere + having to add a lot of HA-related code in the RPC layer. Of course, in the long run, it's more elegant to use XREQ + heartbeat to connect to multiple endpoints and send requests only to those which are alive. But in the short run, the HAProxy broker is dead simple and works very well :-)

by Loic 15y ago

bombela 15y ago

Hi, maintainer of zerorpc-python here. If you want to have to open a dozen of different ZMQ sockets to do HA, do like in the guide. With zerorpc-python, we want to use only the messaging performances of ZMQ and multiplex everything on one single socket (streaming, subrpc!, heartbeat and counting). Note that ZMQ have also different behavior depending of which part of the communication is binding/connecting. On a cloud environment, ports are a precious resource.

Loic 15y ago
Thanks for the comment. If you use the right devices, you do not really need "a dozen" more sockets for HA. But your justification why you prefer HAProxy is what really matters. Every technology is applied in a given context and your context — cloud with a finite number of ports you need not to waste — makes everything clear. Note that I was contacted a week ago to comment on this project and my comment was basically interesting and looks good but this HAProxy thing does not ring ok. So really, add the context in your readme, you will clear a lot of confusion for people used to ZMQ.
- bombela 15y ago
  Added to the already long todo ;) Thanks for the feedback. I also believe that it is not possible to build some sort of connected stream between a DEALER and a ROUTER socket if the DEALER round-robing behind your back. ZMQ 3.0 was giving the tool to use a DEALER in a more router fashioned way, sadly, it got removed in 3.1...
- beagle3 15y ago
  > If you use the right devices, you do not really need "a dozen" more sockets for HA. Would you be able to post a short write up about how to achieve that within the ZeroMQ framework?
  - Loic 15y ago
    Why not, I added it on my list of things to write about. Thank you.

shykes 15y ago

Hi Loic, thanks for the note and it's cool to follow your work from afar. It looks like we get excited about the same stuff :) Disclaimer: I co-founded dotCloud and have the distinctive honor of being the least knowledgeable about zeromq in the entire team. Your comment might have been true 18 months ago - when we first started using zerorpc in production at dotCloud. Since then, we have deployed and scaled hundreds of thousands of applications and I shudder just to imagine how many billions of zeromq messages we have emitted and processed. Believe me we have been through the zeromq guide many times and have experimented with - and abused - many patterns (including majordomo which, as you fail to mention, is not supported out of the box by zeromq and requires a fair amount of custom code of its own). I'm sure zerorpc has many flaws and I know the team looks forward to many constructive debates and - hopefully - patches. But lack of understanding of the zeromq fundamentals, or lack of real-world usage, are 2 things you definitely don't need to worry about :)

Loic 15y ago
Hi Solomon, thank you for your nice comments. Bombela very nicely explained the whys HAProxy, with his explanations everything fall in place nicely (and I must say, I will test drive HAProxy with ZMQ).

jpetazzo 15y ago

It's a deliberate trade-off. ØMQ itself doesn't handle HA automatically. As a client, you can create a REQ socket and connect it to multiple redundant devices. But if a device goes down, you will have to handle timeouts and repeat requests yourself. Additionally, ØMQ will not "kick out" the dead endpoint, so you will end up with 1 out of N requests timing out. So you have to add a decent amount of code in your RPC layer to make everything work properly. And you need to run ZMQ devices anyway. Running a local HAProxy with a 5 lines configuration is, IMHO, simpler than running an extra device somewhere + having to add a lot of HA-related code in the RPC layer. Of course, in the long run, it's more elegant to use XREQ + heartbeat to connect to multiple endpoints and send requests only to those which are alive. But in the short run, the HAProxy broker is dead simple and works very well :-)