Hi HN, this is a personal project I've been working on for some time. This is an amalgam of several good ideas in core.async [1] and durable-queue [2]. There's

Hi HN, this is a personal project I've been working on for some time. This is an amalgam of several good ideas in core.async [1] and durable-queue [2]. There's a fuller description here: http://blog.shriphani.com/2016/01/25/pegasus-a-modular-durable-web-crawler-for-clojure/ http://blog.shriphani.com/2016/01/25/pegasus-a-modular-durab... [1] https://github.com/clojure/core.async https://github.com/clojure/core.async [2] https://github.com/Factual/durable-queue https://github.com/Factual/durable-queue