3 ms·
I built a similar mechanism using SQS and DynamoDB instead of RabbitMQ and PostgreSQL, respectively. The nice part is it requires no servers for job management.
by gfodor 12y ago
I built a similar mechanism using SQS and DynamoDB instead of RabbitMQ and PostgreSQL, respectively. The nice part is it requires no servers for job management. (Just workers obviously.)
https://github.com/gfodor/skyrunner https://github.com/gfodor/skyrunner
(It's still a bit half baked, but could be useful as a starting point.)
- grosskur 12y agoNice, thank you. I've been looking for exactly this---SQS and DynamoDB seem like a good combination.
- eropple 12y agoYou may not care, but I personally avoid SQS and Dynamo for lock-in purposes. I'd rather run my own systems and have the option of moving to GCE or something later.
- gfodor 12y agoI've found in the past that moving code between different queue and job coordination systems is pretty straightforward. Operationally of course you need to learn the new infrastructure but that is unrelated to fears about vendor lock-in.
- eropple 12y agoDynamo is more of a beef than SQS. I've found that Dynamo usage tends to grow. Like a fungus. Applying sufficient fire to remove it later seems beyond many shops' ability to marshal. (Also, SQS is really weird about latency and places pretty ugly limits on responsiveness.)