4 ms·

With Redis, memcached finally has a contender for most awesome cache... thing. However I'm surprised I don't see more benchmarks. Performance should be roughly

by mcs 16y ago

With Redis, memcached finally has a contender for most awesome cache... thing. However I'm surprised I don't see more benchmarks. Performance should be roughly the same unless there's huge I/O contention for persistent storing.

dlsspy 16y ago
I'm a bit biased, but I'm not as optimistic as some. memcached had a design similar to redis when I first got involved with it a few years ago. It was fine for a lot of systems, but didn't scale to the larger ones. It took a lot of work for us to get it to the point where it could saturate all of the cores you could give it. If anyone can ever produce a benchmark where memcached does not perform better than redis, please file a bug in memcached, or at the very least, start a conversation. The closest I've seen were tests that degraded to client tests. In one case, someone took a very poorly performing memcached client and compared it against a very well-tuned redis client and a couple of relational databases and showed that memcached was the slowest of all. sigh In most of these cases, the test environments perform at the very least an order of magnitude slower than what I get on my development laptop (I have no trouble sustaining 90k writes per second with either of a simple c++ thing I threw together yesterday or my high level java client). On real production-quality machines with good networks, 300-400k ops per second isn't hard to achieve. Even between two old "large" (2 core, 7GB) EC2 nodes, I've been doing 90k ops per second all day (which is somewhere around the point where they start to throttle my bandwidth).
- antirez 16y ago
  Hello dlsspy, I never tested memcached, but I'm really sure about Redis numbers: you can get 150k ops/sec in a decent Linux box without problems, per core. So if you got 4 cores you can get 600k ops/sec, assuming you'll have enough bandwidth to saturate that. I agree with you that if Redis performance is better than memcached there is to fill a bug or start a conversation, as there is something odd. Both are written in C and use multiplexing internally, against the same OSes. The performances should be very similar, or the memcached performances should be better as it supports less features. Edit: p.s. can you please explain why running multiple memcached processes, one per core, was not good enough?
  - dlsspy 16y ago
    I'm certainly not trying to start a fight or anything, but there's a lot of bad testing out there, and people like trumpeting the results of such things. Having more machines reduces the performance gained from multigets. This is often a pretty big performance boost. This is especially useful with synchronous clients that serialize requests across servers (which are very common in many popular languages because popular languages don't seem to encourage proper concurrency). Four times the number of servers will quite often make requests take four times longer. Some of the memcached clients will do additional work to hash the keys using an independent node location key just to ensure locality of related data so the values can be retrieved more efficiently.
    - antirez 16y ago
      Sure, many tests are conducted in a wrong way, I fully agree. About multi-gets, this is how Redis handle this, we have a concept called key tags, that I think it is very similar to the one you described. Basically a key in the form foo{bar} gets hashed in a special way, only the part inside {} is hashed ('bar' in my example). So a client willing to optimize for multi get will use {user0}:name {user0}:surname and so forth. Well, not the perfect example as with Redis in this specific case you could use an Hash that is much more space efficient and will provide locality of related keys automatically.
- lzimm 16y ago
  is it too n00b/out-of-place of me to ask what went in to tuning memcached to do that? i'd love to pick your brain (and possibly eat them... creepy, right?)
  - dlsspy 16y ago
    The general path involves adding some locks around critical sections, then benchmarking until those locks bubble up, then reducing the number of locks, then repeating until your server has very little lock contention. The process is pretty straightforward, but as you walk down the path, you end up with something that looks a bit different from where you started.
    - lzimm 16y ago
      hmm, i'm clearly wrong, but it doesn't seem like there's any reason memcached couldn't be implemented completely locklessly... can't most of everything be done with CAS operations instead of taking out locks?
      - dlsspy 16y ago
        That's likely true. We're separating out storage engines and allowing people to write their own. Your engine doesn't need locks. The more stuff we can do in the core without them, the closer we get to your dream. Lock-free hash tables without GC are kind of hard, but not possible. I'd certainly welcome a lock-free engine if you're working on one. :)
        deleted 16y ago
        [deleted]
        lzimm 16y ago
        funny story, i'm crazy n00b, but i'd love to jump in and do one, seriously. wanna show me the ropes on how to get started and maybe hold my hand a little bit while i do all the grunt work? :)
wanderr 16y ago
Redis isn't a direct competitor to memcached. The great thing about Redis (disk-backed cache) is actually a liability if you're using it like an L2 cache which is how memcached is intended to be used. To be clear: in memcached you can effectively set-and-forget keys and never worry about running out of memory. It uses an LRU to figure out what to throw away when memory gets full. In Redis if you want to achieve that behavior you need to expire your keys, which is another round trip every time. That means expire alone is also not enough, because you could set the key successfully but fail to set the expire. That means you need to also keep track of all your keys and periodically do cleanup. Redis is awesome, but I would definitely not use it to replace memcached except in the cases where I absolutely need the data to survive a reboot and am willing to go through the hassle of manually managing the cleanup of every key.
- antirez 16y ago
  Hello, since 2.0 (RC1 is near now...) we have "SETEX" that is SET+EXPIRE in one operation, and with max-memory directive set when Redis runs out of memory will sample three random keys and will delete the one that is going to expire sooner. So this usage is actually supported. What I think about Redis as a cache is that in many contexts the rich data types and operations supported allow better caching. An example: oknotizie.virgilio.it is a large social news site, for this site I built a Redis cache where the "latest" news IDs are pushed into a Redis list, but they are also added to the MySQL DB that was formerly here. So to paginate the "latest news" page I only use fast LRANGE calls against Redis, but if it will return a short read (the cache is empty since the Redis server was restarted) I'll use instead the MySQL server. This actually never happens and everything goes on Redis usually, but since there is no persistent storage needed in Redis side, it is actually a cache. When a news is deleted I use LREM instead. And so forth.
  - andrewvc 16y ago
    Good to hear, but that algorithm isn't LRU, and LRU makes more sense when backing a cache.
    - 16y ago

by mcs 16y ago

dlsspy 16y ago

I'm a bit biased, but I'm not as optimistic as some. memcached had a design similar to redis when I first got involved with it a few years ago. It was fine for a lot of systems, but didn't scale to the larger ones. It took a lot of work for us to get it to the point where it could saturate all of the cores you could give it. If anyone can ever produce a benchmark where memcached does not perform better than redis, please file a bug in memcached, or at the very least, start a conversation. The closest I've seen were tests that degraded to client tests. In one case, someone took a very poorly performing memcached client and compared it against a very well-tuned redis client and a couple of relational databases and showed that memcached was the slowest of all. sigh In most of these cases, the test environments perform at the very least an order of magnitude slower than what I get on my development laptop (I have no trouble sustaining 90k writes per second with either of a simple c++ thing I threw together yesterday or my high level java client). On real production-quality machines with good networks, 300-400k ops per second isn't hard to achieve. Even between two old "large" (2 core, 7GB) EC2 nodes, I've been doing 90k ops per second all day (which is somewhere around the point where they start to throttle my bandwidth).

antirez 16y ago
Hello dlsspy, I never tested memcached, but I'm really sure about Redis numbers: you can get 150k ops/sec in a decent Linux box without problems, per core. So if you got 4 cores you can get 600k ops/sec, assuming you'll have enough bandwidth to saturate that. I agree with you that if Redis performance is better than memcached there is to fill a bug or start a conversation, as there is something odd. Both are written in C and use multiplexing internally, against the same OSes. The performances should be very similar, or the memcached performances should be better as it supports less features. Edit: p.s. can you please explain why running multiple memcached processes, one per core, was not good enough?
- dlsspy 16y ago
  I'm certainly not trying to start a fight or anything, but there's a lot of bad testing out there, and people like trumpeting the results of such things. Having more machines reduces the performance gained from multigets. This is often a pretty big performance boost. This is especially useful with synchronous clients that serialize requests across servers (which are very common in many popular languages because popular languages don't seem to encourage proper concurrency). Four times the number of servers will quite often make requests take four times longer. Some of the memcached clients will do additional work to hash the keys using an independent node location key just to ensure locality of related data so the values can be retrieved more efficiently.
  - antirez 16y ago
    Sure, many tests are conducted in a wrong way, I fully agree. About multi-gets, this is how Redis handle this, we have a concept called key tags, that I think it is very similar to the one you described. Basically a key in the form foo{bar} gets hashed in a special way, only the part inside {} is hashed ('bar' in my example). So a client willing to optimize for multi get will use {user0}:name {user0}:surname and so forth. Well, not the perfect example as with Redis in this specific case you could use an Hash that is much more space efficient and will provide locality of related keys automatically.
lzimm 16y ago
is it too n00b/out-of-place of me to ask what went in to tuning memcached to do that? i'd love to pick your brain (and possibly eat them... creepy, right?)
- dlsspy 16y ago
  The general path involves adding some locks around critical sections, then benchmarking until those locks bubble up, then reducing the number of locks, then repeating until your server has very little lock contention. The process is pretty straightforward, but as you walk down the path, you end up with something that looks a bit different from where you started.
  - lzimm 16y ago
    hmm, i'm clearly wrong, but it doesn't seem like there's any reason memcached couldn't be implemented completely locklessly... can't most of everything be done with CAS operations instead of taking out locks?
    - dlsspy 16y ago
      That's likely true. We're separating out storage engines and allowing people to write their own. Your engine doesn't need locks. The more stuff we can do in the core without them, the closer we get to your dream. Lock-free hash tables without GC are kind of hard, but not possible. I'd certainly welcome a lock-free engine if you're working on one. :)
      - deleted 16y ago
        [deleted]
      - lzimm 16y ago
        funny story, i'm crazy n00b, but i'd love to jump in and do one, seriously. wanna show me the ropes on how to get started and maybe hold my hand a little bit while i do all the grunt work? :)

wanderr 16y ago

Redis isn't a direct competitor to memcached. The great thing about Redis (disk-backed cache) is actually a liability if you're using it like an L2 cache which is how memcached is intended to be used. To be clear: in memcached you can effectively set-and-forget keys and never worry about running out of memory. It uses an LRU to figure out what to throw away when memory gets full. In Redis if you want to achieve that behavior you need to expire your keys, which is another round trip every time. That means expire alone is also not enough, because you could set the key successfully but fail to set the expire. That means you need to also keep track of all your keys and periodically do cleanup. Redis is awesome, but I would definitely not use it to replace memcached except in the cases where I absolutely need the data to survive a reboot and am willing to go through the hassle of manually managing the cleanup of every key.

antirez 16y ago
Hello, since 2.0 (RC1 is near now...) we have "SETEX" that is SET+EXPIRE in one operation, and with max-memory directive set when Redis runs out of memory will sample three random keys and will delete the one that is going to expire sooner. So this usage is actually supported. What I think about Redis as a cache is that in many contexts the rich data types and operations supported allow better caching. An example: oknotizie.virgilio.it is a large social news site, for this site I built a Redis cache where the "latest" news IDs are pushed into a Redis list, but they are also added to the MySQL DB that was formerly here. So to paginate the "latest news" page I only use fast LRANGE calls against Redis, but if it will return a short read (the cache is empty since the Redis server was restarted) I'll use instead the MySQL server. This actually never happens and everything goes on Redis usually, but since there is no persistent storage needed in Redis side, it is actually a cache. When a news is deleted I use LREM instead. And so forth.
- andrewvc 16y ago
  Good to hear, but that algorithm isn't LRU, and LRU makes more sense when backing a cache.
  - 16y ago