4 ms·
Show HN: Redis-pipe – Use Redis lists like Unix pipes
- waitingkuo 12y agoWhat's the benefit to use Redis pipe instead of Unix pipes?
- bytesandbots 12y agoI guess it might be very useful in a cloud like environment where you use this set up to pipe data across scaled up servers.
- morgenkaffee 12y agoI use it as a poor man's job queue. cat jobs.txt | redis-pipe jobs An then on several workers: redis-pipe --count 10 jobs | python do-work.py | redis-pipe results An then in the end: redis-pipe results > results.txt Or you can use it as simple logging mechanism: tail -f /var/log/syslog | ./redis-pipe logs
- eddd 12y agoPoor man's job queue is celery with redis backend i guess.
- morgenkaffee 12y agoMine is even more poor :)
- nissimk 12y agoWhy are you using LPUSH/LPOP instead of PUBLISH/PSUBSCRIBE? I guess you get storage that way, but only one consumer of each message. It would be nice if it was optional in case you had a different application where you need to have multiple subscribers to the same topic.
- morgenkaffee 12y agoYes you're right. I can create a flag --publish and --subscribe.
- fprotthetarball 12y agoSee this for more inspiration (using pub/sub): https://github.com/whee/rp https://github.com/whee/rp
- morgenkaffee 12y agoAh that's the same idea! Wished I found that before. Thanks man.
- fprotthetarball 12y agoThe interesting thing about rp is that you can use it as a pipeline in your other Go applications, as the Redis pub/sub "pipe" is exposed as a normal Go io.Reader/io.Writer interface. Might be something worth expanding on somehow.