5 ms·
I am continually amazed by Redis. I am involved in a project where parallel processing and work distribution is achieved via 0mq and Redis acts as a sort of "sh
by i_am_a_human 15y ago
I am continually amazed by Redis. I am involved in a project where parallel processing and work distribution is achieved via 0mq and Redis acts as a sort of "shared memory". The performance is outstanding, and it is a fun architecture to work in.
- SoftwareMaven 15y agoI would love to hear more about that architecture. I just added 0MQ into our product to handle pushing and pulling events, among other things. Some of the events are OK if they get lost; others, not so much so. I've been thinking about adding something like redis or mongodb to the mix to provide a layer of persistence to those messages.
- i_am_a_human 15y agoWe're not necessarily using Redis to persist the 0mq messages, but more so allowing parallelized processes a medium to share data. The single threaded nature of Redis and 0mq work distribution eliminate a lot of nasty concurrency issues. You may want to look into the Redis pub/sub though for what you are describing (http://redis.io/topics/pubsub http://redis.io/topics/pubsub)
- gmt2027 15y agoI am also working on parallel processing for my Phd and implementing a Redis-based distributed shared memory queueing system that looks a lot like a tuple space. The Redis primitives fit this messaging model perfectly, allowing very loose coupling between language-neutral computational elements. I do have concerns that the single-threading may eventually impose a bottleneck with the scale of deployment intended (1000s of CPUs and GPUs). I have been looking into 0MQ to overcome these limitations and wonder how feasible it would be to implement a 0MQ socket connection option for Redis.
- dochtman 15y agoWe use 0MQ and Redis together at work. We have an incremental flow of updates that we distribute internally over 0MQ pubsub, and have one listener process that updates the full state in Redis from each update message. That way, we have both (a) a way to easily get the latest state (and fast), and (b) an easy way to listen for updates to the state. It works beautifully, even with a high rate of updates.