15 ms·

Faktory is CouchDB, Machinery is Redis/Memcache/MongoDB, Sandglass is a custom Raft on top of RocksDB… I'd love to see comparative stress tests, Jepsen-like, t

Faktory is CouchDB, Machinery is Redis/Memcache/MongoDB, Sandglass is a custom Raft on top of RocksDB… I'd love to see comparative stress tests, Jepsen-like, to assess the ability to survive partitions, corruption, node restart and node loss, to better estimate the probability that a job gets lost.

richardknop 9y ago
Machinery is basically a Go implementation of celery (popular Python task queue). It has two core components: broker (AMQP or Redis supported) and backend (Memcache, Redis or MongoDB or even no backend if you don't care about storing task states). I think comparison with Sandglass would not be valid as Machinery is a higher level job/task queue while Sandglass is a lower level library (basically a message queue such as RabbitMQ which would be just a component of Machinery). Faktory vs Machinery could be compared as they are on more or less same level of abstraction.
agnivade 9y ago
Faktory is based on rocksdb actually.