4 ms·
True, but with NSQ being brokerless, the lines are somewhat blurred between client and server. Each of our app servers has its own local nsqd. If that nsqd sto
by mnutt 12y ago
True, but with NSQ being brokerless, the lines are somewhat blurred between client and server.
Each of our app servers has its own local nsqd. If that nsqd stops responding, we take the app server out of the load balancer. The local nsqd publishes to multiple channels, which get consumed by hosts in different datacenters.
There are still potential failure cases: nsqd loses connection to consumers, messages start building up, then the machine somehow goes away. The only way to prevent it is to take the machine out of the load balancer any time nsqd messages start backing up, but we prioritize serving requests over sending messages.
- arunoda 12y agoYes. I didn't want to rant our NSQ. This is the reason we choose NSQ because of it's simplicity. We maintain NSQ close to the consumer. We don't our publisher to take responsibility to the message processing. Once it's push to the queue, we need to make sure it's getting process anyway. That's why we publish same message to multiple queues. All our DB operations are idempotent. So, we are okay with processing the same message multiple times.