5 ms·
Scaling NSQ to 750B Messages
- 0xmohit 10y agoAwesome. As someone who has been using it in production for a while now, I can say NSQ rocks. http://nsq.io/talks_and_slides.html http://nsq.io/talks_and_slides.html contains interesting links too.
- saintfiends 10y agoNSQ is a cool piece of software. I've used it to distribute work. One thing that I miss from other queue systems are that it doesn't guarantee dedupe message delivery. Is it still the case?
- 0xmohit 10y agoYes [1]. This means, for a variety of reasons, messages can be delivered multiple times (client timeouts, disconnections, requeues, etc.). It is the client’s responsibility to perform idempotent operations or de-dupe. http://nsq.io/overview/features_and_guarantees.html http://nsq.io/overview/features_and_guarantees.html
- tveita 10y agoWhat queue systems guarantee non-duplicated message delivery? At-least-once and at-most-once delivery can done with relatively easy hand-off strategies, but I don't see how you would guarantee exactly-once delivery between the message queue and your application.
- saintfiends 10y agoYou're right, none that I know does but I don't mean exactly-once. I meant I like NSQ but sometimes I do wish they had a fire-and-forget mode.
- yahyaheee 10y agoCan this replay messages like Kafka?
- ryanjshaw 10y agoIt doesn't seem like it. The article specifically talks about replacing some NSQ-based workers with Kafka-based workers. I would love to hear whether anybody uses in-memory messaging systems in a configuration that has guarantees similar to those Kafka offers, or is it simply not possible?
- otterley 10y agoIt's not possible, if only because memory-backed queues are not durable by definition.