mostly just scale. Keeping your queue in memory isnt necesarilly a requirement unless you have a lambda arch or a realtime requirement (which we started with).

mostly just scale. Keeping your queue in memory isnt necesarilly a requirement unless you have a lambda arch or a realtime requirement (which we started with). For our larger operations, kafka, which is disk backed, distributed with a little more ease and cheaper was a nice option. For reliability, we were using some of storms primitives as well as the mechanisms inside qless, the queuing library we were using on top of redis. https://github.com/ChannelIQ/qless-java https://github.com/ChannelIQ/qless-java