4 ms·

Redis Cluster: a pragmatic approach to distribution

42 points by mace 16y ago

owyn 16y ago
Interesting, but who wrote this? Is it a design doc for redis cluster?
- antirez 16y ago
  yes, actually is at best an overview since many important details are missing (the presentation time was 20 minutes),a a few more are still a work in progress. Currently implemented in my cluster branch: MIGRATE, and the basic PING/PONG with gossip. I'm working at the rest full time, so I hope to have some alpha to show in two months at max.
- mace 16y ago
  Yes, this is antirez's presentation about Redis cluster at a recent Redis meetup: http://twitter.com/antirez/status/28939763430 http://twitter.com/antirez/status/28939763430
robbiehudson 16y ago
We will be posting a video of the talk that Salvatore gave in the next few days.
amix 16y ago
I am a big fan of Redis, but I have mostly used it for smaller databases. With Redis VM and upcoming Redis Cluster I simply can't imagine why one would not use Redis as a more central database, since it will offer great features, great perfomance and great scalability.
- jpeterson 16y ago
  I simply can't imagine why one would not use Redis as a more central database Here's my main reason: Redis is still very new and unproven. I'd feel extremely uneasy using Redis as a store for mission-critical data, and I can't really justify adding complexity to my server configuration for features that are not mission-critical.
jbl 16y ago
I'm curious about how well the all-to-all communication strategy scales as clusters grow large. Just going off of this slide, the number of links and amount of communication grows pretty fast as nodes are added. Nodes could get inundated with management data... OTOH, maybe there's enough computing resources in a modern physical box to deal with this, particularly if you run a dedicated network for inter-node communication.
- thomasmeeks 16y ago
  I remember some enterprise java cache systems running into problems with an all-to-all communication model. It is a problem you can solve with hardware (second network as you mentioned), but can be a limiting factor for sure. 4,096 nodes means 8,386,560 connections between them, and the associated bandwidth caused by the nodes checking up on each other.
  - simonw 16y ago
    I don't think Redis cluster quite works like that - it's a gossip system, so each Node only needs to talk to a few other nodes directly to get information about a much larger number of Nodes.

42 points by mace 16y ago

owyn 16y ago

Interesting, but who wrote this? Is it a design doc for redis cluster?

antirez 16y ago
yes, actually is at best an overview since many important details are missing (the presentation time was 20 minutes),a a few more are still a work in progress. Currently implemented in my cluster branch: MIGRATE, and the basic PING/PONG with gossip. I'm working at the rest full time, so I hope to have some alpha to show in two months at max.
mace 16y ago
Yes, this is antirez's presentation about Redis cluster at a recent Redis meetup: http://twitter.com/antirez/status/28939763430 http://twitter.com/antirez/status/28939763430

robbiehudson 16y ago

We will be posting a video of the talk that Salvatore gave in the next few days.

amix 16y ago

I am a big fan of Redis, but I have mostly used it for smaller databases. With Redis VM and upcoming Redis Cluster I simply can't imagine why one would not use Redis as a more central database, since it will offer great features, great perfomance and great scalability.

jpeterson 16y ago
I simply can't imagine why one would not use Redis as a more central database Here's my main reason: Redis is still very new and unproven. I'd feel extremely uneasy using Redis as a store for mission-critical data, and I can't really justify adding complexity to my server configuration for features that are not mission-critical.

jbl 16y ago

I'm curious about how well the all-to-all communication strategy scales as clusters grow large. Just going off of this slide, the number of links and amount of communication grows pretty fast as nodes are added. Nodes could get inundated with management data... OTOH, maybe there's enough computing resources in a modern physical box to deal with this, particularly if you run a dedicated network for inter-node communication.

thomasmeeks 16y ago
I remember some enterprise java cache systems running into problems with an all-to-all communication model. It is a problem you can solve with hardware (second network as you mentioned), but can be a limiting factor for sure. 4,096 nodes means 8,386,560 connections between them, and the associated bandwidth caused by the nodes checking up on each other.
- simonw 16y ago
  I don't think Redis cluster quite works like that - it's a gossip system, so each Node only needs to talk to a few other nodes directly to get information about a much larger number of Nodes.