3 ms·
> Oban allows you to insert and process jobs using only your database. You can insert the job to send a confirmation email in the same database transaction wher
by simonw 10mo ago
> Oban allows you to insert and process jobs using only your database. You can insert the job to send a confirmation email in the same database transaction where you create the user. If one thing fails, everything is rolled back.
This is such a key feature. Lots of people will tell you that you shouldn't use a relational database as a worker queue, but they inevitably miss out on how important transactions are for this - it's really useful to be able to say "queue this work if the transaction commits, don't queue it if it fails".
Brandur Leach wrote a fantastic piece on this a few years ago: https://brandur.org/job-drain https://brandur.org/job-drain - describing how, even if you have a separate queue system, you should still feed it by logging queue tasks to a temporary database table that can be updated as part of those transactions.
- airocker 10mo agoDebezium was built exactly for that to power a queue based on WAL.
- nhumrich 10mo agoThis is called the "transactional outbox pattern"!
- simonw 10mo agoGood name! Looks like SeatGeek use that naming convention here: https://chairnerd.seatgeek.com/transactional-outbox-pattern/ https://chairnerd.seatgeek.com/transactional-outbox-pattern/ This looks like a good definition too: https://www.milanjovanovic.tech/blog/outbox-pattern-for-reliable-microservices-messaging https://www.milanjovanovic.tech/blog/outbox-pattern-for-reli...
- sieep 10mo agoExcellent point. Never thought of transactions in this way.
- zie 10mo agoI agree this is an awesome feature. I use pg_timetable instead of Oban though: https://cybertec-postgresql.github.io/pg_timetable/v6.x/ https://cybertec-postgresql.github.io/pg_timetable/v6.x/
- elfly 10mo agoLots of people are also trying to just use postgres for everything, tho.
- nichochar 10mo agoWe're building an AI powered app builder. We use elixir, phoenix and of course OBAN. It feels like such a super power. What you're describing is particularly important in the era of long running AI processes. Something as simple as running a deploy creates pressure on your agent orchestration. But if everything is powered by OBAN you have wonderful ways to build durability. By the way, it's all "free" and built-in. In other language ecosystems, people literally pay for durability that has this shape (like temporal)
- peheje 10mo agoThis resonates so much. I spent years in an org watching "domain events" vanish into the ether because of the Dual Write Problem. We had these high-performance, sharded, distributed monsters that were "fast" on paper, but they couldn't guarantee a simple message would actually send after a record was saved. Moving back to a rock-solid SQL-backed approach solved it overnight. But since there are no more "1% glitches," people have forgotten there was ever a fire. It’s a thankless win. The organization now thinks the system is "easy" and the "async purists" still lobby for a separate broker just to avoid "polluting" the DB. They’d rather trust complex, custom-built async logic than the most reliable part of their stack. (The transactional outbox pattern is essential, I just prefer mine backed by the same ACID guarantees as my data). It’s tricky stuff. I'm an application dev, not a DB internalist, but I've realized that a week spent actually learning isolation levels and commit-ordering saves you a year of "distributed system" debugging. Even when teams layer an ORM like Entity Framework on top to "hide" the complexity, that SQL reality is still there. It’s not magic; it’s just ACID, and it’s been there the whole time.