4 ms·

Another interesting alternative is to put only "hot" data (metadata) on Redis, and use another on-disk DB for the rest. Yup that's how I (still) see redis at t

by moe 17y ago

Another interesting alternative is to put only "hot" data (metadata) on Redis, and use another on-disk DB for the rest. Yup that's how I (still) see redis at the moment, more a persistent cache than a primary datastore. EC2 is just expensive from every angle you see it Not really, but that's a different story (scaling out vs up etc.). In general redis as of now is mostly geared towards a "scale-up" approach whereas cloud deployments naturally need to "scale-out" instead. Application level partitioning is a good option I think. It's easy to manage, for instance you take your users in an instance, their blog posts in another, and comments in another one. Are you little and low traffic in the start and can't buy three hosts? Just use three Redis instances in the same box, and move them in different hosts as you grow. Yes, and that's exactly the can of worms you don't want to get into. What happens when one of your comment/user/post instances outgrows its host? You have to split it further. Either logically again (users A-L on host1, M-Z on host2) or anonymously (even users on host1, odd users on host2). Since nobody wants to constantly think about their data-layout the latter variant is definitely preferable. MongoDB has a leg up here by providing a beta of anonymous, maintenance-free sharding already. This will only work well if data access pattern is biased btw. And the virtual memory will not remove completely the dataset size limitation. For instance if you have 2 GB of RAM it will make sense to setup VM to have up to 32 GB of data for instance, given a biased enough access pattern, but it's not like it will work well with 1 TB of data. That sounds bad. Very bad. Not for the people who are happy with redis today, but for those who are not touching it because of that constraint. There are solutions to distribute among different servers that are not hard to implement and maintain. Okay, I'll bite. Where is the turnkey solution that distributes my data over n redis-instances with automatic failover, automatic rebalancing after adding/removing an instance, n-copies for redundancy, reliable failure modes when an instance outgrows the available memory? I would say without at least some of these features a redis-cluster could become a nightmare to maintain in the long run.

antirez 17y ago
I would say without at least some of these features a redis-cluster could become a nightmare to maintain in the long run. Indeed, this is very helpful, and it's exactly what redis-cluster will do. You talk to Redis-cluster, and it will talk to other N redis instances, handling faults, adding or removing nodes, and so forth. It's like "mongos" process basically. But the roadmap is to implement virtual memory first and redis-cluster later, as I think that virutal memory is a more promptly available solution to start, and works with applications designed to run into a singel Redis server (think to SORT BY and so o, sets intersections and so forth).

by moe 17y ago

antirez 17y ago

I would say without at least some of these features a redis-cluster could become a nightmare to maintain in the long run. Indeed, this is very helpful, and it's exactly what redis-cluster will do. You talk to Redis-cluster, and it will talk to other N redis instances, handling faults, adding or removing nodes, and so forth. It's like "mongos" process basically. But the roadmap is to implement virtual memory first and redis-cluster later, as I think that virutal memory is a more promptly available solution to start, and works with applications designed to run into a singel Redis server (think to SORT BY and so o, sets intersections and so forth).