7 ms·
Redis Cluster Re-Implemented in Rust: Scaling Redis Easily in Kubernetes
- DeathArrow 5y agoI might have an use for it since we will have to expand our Redis instances to clusters at work.
- doyoubi 5y agoFeel free to let me know if you get any questions :)
- gizdan 5y agoI don't know a whole lot about Redis, so I'm struggling to understand what this is. Am I correct in saying that this is just the clustering mechanism? So I'd need to run Redis instances, and use this to cluster them? If so, would this work with KeyDB (would that even make sense)?
- doyoubi 5y agoYes, that's accurate. This project runs on top of Redis as Redis has already done a good job as a memory storage engine. And https://github.com/doyoubi/undermoon-operator https://github.com/doyoubi/undermoon-operator will help run it with Redis instances together, so you won't need to set them up manually. And yes, it can work with KeyDB in theory.
- deleted 5y ago[deleted]
- lrem 5y agoIs Redis still single-threaded? I thought that would become a bit of a problem by now.
- doyoubi 5y agoRedis 6 has already supported multiple threads for IO. The memory storage engine is still inside a single thread but I think it's fine.
- lrem 5y agoDo you mean that a single CPU can saturate the throughput of a machine's RAM? That's a bit surprising, given that current crop of Intel CPUs does non uniform memory access. Or, is that bandwidth so much above any I/O that it doesn't matter?
- doyoubi 5y agoNop. It's that a better solution would still be using multiple Redis instances on each physical server. Trying to support multi-threads in the storage engine requires complex synchronization implementation. Sharing nothing between each small instance achieves higher performance and is a lot simpler. Also, FULL SYNC between masters and replicas can be uncontrollable. And smaller memory size for each instance alleviates the performance impact from FULL SYNC.
- ohthehugemanate 5y agonot to mention, multiple instances follows the indistry direction in horizontal scalability. Common advice when building for k8s style cloud environments is to target single core, n instances, to maximize scaling flexibility and atomicity.
- throwaway81523 5y agoThere is a parallel but less fancy reimplementation of Redis called Pedis: https://github.com/fastio/1store https://github.com/fastio/1store It's built using Seastar, which is sort of an ugh but cool unholy marriage of node.js and C++14 with an event loop per thread, and message passing between threads. It predates C++20 coroutines I guess, so you pass callbacks around like in Node using C++ lambdas. Maybe they have updated that by now. They have also done a version of memcached using DPDK. I don't see a Pedis one, but check out their memcached one (scroll down): http://seastar.io/ http://seastar.io/
- Fiahil 5y agoHow does it work ? Is it sharding different keys to different Redis instances ?
- doyoubi 5y agoYes! It uses exactly the same slot model in the official Redis Cluster. Each key will first calculate its slot number by a hashing function. Then try to find out the instance with this corresponding slot. So this solution is compatible with the existing Redis Cluster clients from the community.
- Fiahil 5y agoMakes perfect sense :) This allow scaling without impacting the consistency guarantees of redis !
- nahname 5y agoThe master/replica setup is interesting because we did something similar at Shopify, but I don't believe this is conventional for redis cluster. Can you expand on that a bit? eg: Looks like master/replica are together on a host, wouldn't you want them spread out? You are including proxies for clients and server? How do they compare to envoy? eg: Are you doing anything clever to handle failovers transparently for clients? Is the metadata storage just using redis?
- doyoubi 5y agoI have to admit the diagram is a bit misleading. The core idea here is called chunk: https://github.com/doyoubi/undermoon/blob/master/docs/chunk.md https://github.com/doyoubi/undermoon/blob/master/docs/chunk.... Each proxy always runs in front of 2 Redis instances. Each chunk consists of 2 proxies and thus 4 Redis instances on 2 different physical servers. And each Redis has its master/replica on another half part under another proxy inside the chunk. So master and replica are always not in the same physical server. Chunk is the smallest building block of a cluster so the Redis instances number is always a multiple of 4. We designed it in this way because we came up with an algorithm to balance the overall workload based on this chunk structure: https://github.com/doyoubi/undermoon/blob/master/docs/chunk_allocation.txt https://github.com/doyoubi/undermoon/blob/master/docs/chunk_... For the second question, the proxy here is just a way to add additional features on the top of Redis without changing its source code. I would say it's a server-side proxy while envoy focuses on the client-side features. And yes, it can handle failover transparently for clients. In Kubernetes, the metadata is stored inside a ConfigMap, which is based on etcd.
- djbusby 5y agoLike Tendis? https://github.com/Tencent/Tendis https://github.com/Tencent/Tendis Tendis is a high-performance distributed storage system fully compatible with the Redis protocol.
- doyoubi 5y agoTendis focuses more on persistence and it uses decentralized architecture just as Redis Cluster.