3 ms·
The use case I'm referring to is when the client must be sharded to avoid CPU or network or disk bottlenecks.
by itaifrenkel 9y ago
The use case I'm referring to is when the client must be sharded to avoid CPU or network or disk bottlenecks.
- manigandham 9y agoThen that's exactly what consumer groups help with but it sounds like you want partitioning then - which is exactly what Kafka does but with a little more automation. Run multiple Redis instances and use a simple hash based on whatever key you want to route messages and get the throughput you need. It's probably never going to be part of the core Redis logic but should be possible as a module to do the routing when used in a Redis cluster.
- itaifrenkel 9y agoWhat I'm referring to is changing the partitioning dynamically by splitting streams, not just redis nodes. Here is one implementation example http://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-split.html http://docs.aws.amazon.com/streams/latest/dev/kinesis-using-... Doing it without server support is tricky.