4 ms·
I'm a big fan of RabbitMQ ( http://www.rabbitmq.com http://www.rabbitmq.com ). It's an AMQP message queue and has a bunch of very nice features like: - Persist
by bjpirt 16y ago
I'm a big fan of RabbitMQ ( http://www.rabbitmq.com http://www.rabbitmq.com ). It's an AMQP message queue and has a bunch of very nice features like:
- Persistent queues
- Clustering
- Topic based queues for an extremely flexible syndication model
- Transactional (acknowledged) queues
- Lots of adapters in different languages
- Fast, handles thousands of messages/sec
It's written in erlang so if you're already running CouchDB then you can maximise any language specific knowledge.
Replicated queues are tricky to set up with any queuing system, but if you're looking to a) not lose any messages and b) have high availability then the simplest thing is just to run 2 independent but identically configured queues in parallel which are configured to be persistent and configure the workers to pull jobs of each queue in turn. If one of them goes down you can restart it and recover the lost jobs whilst the other queue continues.
Alternatively you can write to both queues and deduplicate the messages when you pull them off. Beetle ( http://xing.github.com/beetle/ http://xing.github.com/beetle/ ) takes this approach and uses Redis to manage the deduplication.
- ifesdjeen 16y agoAgreed! When using RabbitMQ / 0MQ or even Redis for queue replacement it's possible to build a great infrastructure around them, you're not sticking to the worker library launcher set. You simply have a queue, you may distribute it and get it working the way you want to.