3 ms·
Aside from that, the main advantage of this is transactions. I can do: begin; insert_row(); schedule_job_for_elasticsearch(); commit; And it's gua
by arp242 2y ago
Aside from that, the main advantage of this is transactions. I can do:
begin;
insert_row();
schedule_job_for_elasticsearch();
commit;
And it's guaranteed that both the row and job for Elasticsearch update are inserted.
If you use a dedicated queue system them this becomes a lot more tricky:
begin;
insert_row();
schedule_job_for_elasticsearch();
commit; // Can fail, and then we have a ES job but no SQL row.
begin;
insert_row();
commit;
schedule_job_for_elasticsearch(); // Can fail, and then we have a SQL row and no job.
There are of course also situations where this doesn't apply, but this "insert row(s) in SQL and then queue job to do more with that" is a fairly common use case for queues, and in those cases this is a great choice.
- nostrebored 2y agoMost of these two phase problems can be solved by having separate queue consumers. And as far as I can tell, this is only a perk when your two actions are mutate the collocated database and do X. For all other situations this seems like a downgrade.
- jashmatthews 2y agoDo you mean like the consumer for the first phase enqueues a job for the second phase?
- jashmatthews 2y agoTransactional Outbox solves this. You use a table like in the first example but instead of actually doing the ElasticSearch update the Outbox table is piped into the dedicated queue.