2 ms·
Besides the distributed case, message bus is invaluable in building crash-proof applications. I've used lightweight message bus within the app itself, for bett
by ww520 7y ago
Besides the distributed case, message bus is invaluable in building crash-proof applications. I've used lightweight message bus within the app itself, for better crash recovery on long running tasks. E.g. Need to generate lots of emails and send them out. I would create a small command object for each email and queue it to the message bus, and let the message handler to handle email generation one by one. The lightweight command objects can be queued up quickly and the UI can return to the user right the way. The slow running email generation can be run in the background. In the event of a system crash, the message bus will be restored automatically and continue on the remaining tasks in the queue.
- sk5t 7y agoWhat if your process to generate the initial messages crashes halfway through?
- ww520 7y agoYou return the error to the user. Crash handling is a matter of managing user expectation. If the user hits a button and the UI shows the command as success, the system better ensures the command will complete eventually. If the user hits a button and the UI shows an error, well the user can try again when the system is up.
- sk5t 7y agoAh, let me restate the question: suppose the user has clicked a button to approve sending 5,000 emails; the server process puts 2,000 messages onto a queue for later processing--but then, something occurs to interrupt adding the remaining 3,000 messages to the queue. Presuming the queue doesn't support any transactionality beyond per-message, and optionally supposing a queue consumer has already started sending some of these emails, how do you recover from the fault and help the user send the rest of the email (without duplicating outbound email)?
- hderms 7y agoImo I'd resort to an ACID datastore if permitted but I'm curious if there are common patterns that enable this with messaging systems.
- mattcaldwell 7y agoYou update the status of each message when it's acked. You show a live count of messages sent / total messages on the screen where the user sent it. 2000/5000 sent... if those 3,000 never get sent, it will be obvious to the user. If you want the user to be able to try re-sending, you can provide that functionality... you'd need to "cancel" the outgoing messages using a separate queue, re-sending when each cancellation is acked.
- sk5t 7y agoIt seems as though you might be describing a process with state stored in an RDBMS or the like. Which, while a perfectly reasonable approach, is not much like the initially-described case of firing a bunch of "send email to foo@example.com"-type messages into a queue, subsequently to be drained and acted upon by possibly-remote workers. What I am trying to uncover here is how one might expect to use a queue, on its own, to support a very-much-non-idempotent interruptible non-transactional process. Also, what does it mean to "cancel" an already-sent email?
- ww520 7y agoQueue is a component of the solution for this kind of non-transactional process, not the entire thing.
- floriol 7y agoI have yet to use it, but isn't Java EE's Batch API is used for things like this? It tracks the progress of long running tasks (e.g. the index of the current mail) and can continue from the last sent index in case of some error.
- tedeh 7y agoI've solved this problem by adding a status field to my email objects/rows which can assume the values "Unsent, sending, sent, error". I guess it is a bit self-explanatory, but status values are set like this: - Unsent: Default status - Sending: Message dispatched to broker - Sent: Send task/message ack'ed - Error: Send task/message any other exception This of course assumes that your 5 000 emails are not ephemeral and are in a database. If by "doesn't support any transactionality" you mean ack's aren't possible for one reason or the other then of course you have to go a slightly different route (pun somewhat intended) by publishing a "result" task/message and updating your database rows based on what comes back on this new "task_result" queue.