4 ms·
Is there a similar library in Python? I really like the look of this approach, but don't want to build it myself if I can avoid it.
by theptip 7y ago
Is there a similar library in Python?
I really like the look of this approach, but don't want to build it myself if I can avoid it.
- zrail 7y agohttps://pypi.org/project/pq/ https://pypi.org/project/pq/ looks similar, although it looks like they use `SKIP LOCKED` instead of advisory locks. I'm not sure what the tradeoff is.
- fjp 7y agoSKIP LOCKED is much simpler and doesn't really have any gotchas. We had an internal library that used advisory locks which had all sorts of strange behavior we couldn't figure out until we just moved to SKIP LOCKED
- faizshah 7y agoI was looking into python task queuing recently, here's some other choices from my notes: https://github.com/malthe/pq https://github.com/malthe/pq - postgres queue based on rq and ruby queue_classic https://github.com/coleifer/huey https://github.com/coleifer/huey - sqlite, redis and in memory by coleifer (peewee creator). Possible to implement postgres storage layer simply. https://github.com/closeio/tasktiger https://github.com/closeio/tasktiger - flexible redis-based python task queue alternative to celery https://dramatiq.io/ https://dramatiq.io/ - actor based python job queue on redis/rabbitMQ https://github.com/GoogleCloudPlatform/psq https://github.com/GoogleCloudPlatform/psq - gcp pub/sub based task queue
- theptip 7y agoI just remembered that Celery has a DB backend too, but I've not looked deeply; I can't see it doing anything particularly fancy in the implementation: https://github.com/celery/celery/blob/master/celery/backends/database/__init__.py https://github.com/celery/celery/blob/master/celery/backends...