3 ms·
One thing I've noticed with my own Clojure service at scale, is that eventually, I want to significantly reduce my use of futures entirely. When you're getting
by arohner 13y ago
One thing I've noticed with my own Clojure service at scale, is that eventually, I want to significantly reduce my use of futures entirely.
When you're getting started with Clojure, it's easy to do everything that needs to happen "later" or "in the background" as a future. Once you go to production, you realize that about half of those futures should have been jobs going into a queue somewhere, with logging and guaranteed delivery, retrying on another machine, if necessary.
I haven't gotten to the point of writing that library, but I probably will, soon.
- prospero 13y agoHey Allen, at Factual we're working on something in that vein: https://github.com/factual/skuld https://github.com/factual/skuld. As you might expect, doing it right takes something a bit more ambitious than just a library. If you're interested in collaborating, or just have thoughts on how it could be more useful for you, let me know.
- adambard 13y agoI've had great success using Lamina for in-process message queues in lieu of futures, although this does nothing towards making your system distributed.
- cgag 13y agoIn process queues are a good abstraction, but that doesn't really protect you in case of your app crashing, which I think is the concern more than being distributed. I may be misunderstanding though. Edit: I was already a big fan of SoundCloud, very pleased to find out they use Clojure.
- prospero 13y agoI think the point was that it's hard to make something failure-tolerant without being distributed. A locally persisted queue would avoid certain failure modes, though.
- josephwilk 13y agoYes, I shed a tear as I had to remove all the futures. Disappointment I could not get multiple thread pools and scheduling with Clojure's concurrency primitives.