7 ms·
This ensures that the only edge case that would result in message loss is an unclean shutdown of an nsqd process. In that case, any messages that were in memory
by bgentry 14y ago
This ensures that the only edge case that would result in message loss is an unclean shutdown of an nsqd process. In that case, any messages that were in memory (or any buffered writes not flushed to disk) would be lost.
I don't understand how you can call it a message delivery "guarantee" when you're susceptible to losing messages when a node dies.
One solution is to stand up redundant nsqd pairs (on separate hosts) that receive copies of the same portion of messages.
OK, delivery is only guaranteed if I run multiple independent sets of NSQd and write messages to both.
Regardless, it looks like an interesting project.
- jlgreco 14y agoHow would you design a system that didn't have this problem at the individual node level?
- dclusin 14y agoSequence numbers on messages sent by producers coupled with some sort of persistence by the sender. This way when a gap is detected some sort of resend protocol can be implemented to achieve the delivery guarantee.
- philjr 14y agoA similar design to TCP/IP - sequence numbers, acknowledgements & resending capability on sender side
- onetwothreefour 14y agoThis is a solved problem in most messaging protocols.