7 ms·

Memcached 1.6

110 points by nrr 7y ago

haolez 7y ago
I have a trading bot that consolidates a lot of price information in order to make decisions. I'm currently using Redis due to data structures (lists and scores) and Pub/Sub. I wonder if memcached would give me performance benefits and lower my CPU costs.
- manigandham 7y ago
  CPU is cheap and memcached has no advanced data structures. If you're using them then you'll probably have to reimplement everything yourself in memcached with less efficiency.
  - henriquez 7y ago
    You can always just JSON-encode whatever you’re putting in there and decode on the way out. Though I couldn’t imagine using it for pub/sub like the parent commenter is doing with redis.
    - manigandham 7y ago
      Which is inefficient and pointless compared to just using Redis with data structures that you can manipulate directly.
- dormando 7y ago
  Mind giving some examples in more detail about how you make use of lists/scores?
  - haolez 7y ago
    Pub/Sub is easily replaceable. I use lists and scores to store orders from orderbooks, where the prices are the scores. Then I can order by price. It's simple, but I have like 400 updates per second.
- jasonwatkinspdx 7y ago
  You'll get a bunch of garbage answers here to be frank, from people who never put more than trivial load through both. Memcache blows redis out of the water in terms of raw throughput and latency, like it's not even remotely close on modern lots of cores and big memory machines. Redis has a more feature rich api, but has fundamentally hobbled itself on a root hash table and concurrency implementation that is circa 1990. Give memcached a try, even if it means you implement some stuff you currently defer to redis ops. Measure it. Look at the code bases. Make up your own mind about performance and which actually makes your code simpler.
  - dijit 7y ago
    http://antirez.com/news/94 http://antirez.com/news/94 TL;DR: Redis is just doing different things, and the slab allocator is pretty efficient in regards to performance but not memory usage.
    - dormando 7y ago
      Fwiw this post comparison is pretty old. The memory usage should be the same or better these days (automatic slab rebalancing, LRU expired item crawler, chunked items, etc).
  - haolez 7y ago
    Thanks. I'll make sure to create a PoC and post back here.
dleslie 7y ago
This was the first non-sql datastorage I used for online services; a lifetime ago. Glad to see that it is still maintained.
- qeternity 7y ago
  I wouldn't be surprised if there is more data cached by memcached than redis with all the massive players using. Hardly a dead project.
- henriquez 7y ago
  I still use Memcache in a handful of old PHP projects I’ve been keeping on life support since the mid 2000s. I’m also glad it’s being maintained! It’s perfectly fine at what it was designed to do.
coleifer 7y ago
Memcached doesn't pander or resort to marketing to achieve a userbase (compare to redis/labs). It's a great piece of software engineering and I absolutely respect that there's no gimmick, no bullshit. Just a lot of hard work given away for free. Great work!
- dormando 7y ago
  Thanks!
  - cgidriver 7y ago
    Thanks for all your efforts! It is indeed a very useful piece of software. Easy to install and use, well documented, and very helpful. Is there documentation to run it distributed somewhere you can point me to?
- vosper 7y ago
  > Memcached doesn't pander or resort to marketing to achieve a userbase (compare to redis/labs) I think this is pretty unfair to Redis. Redis has a lot more features (tell me which ones you think are pandering, and to whom?), is free for the vast majority of what people want to use it for, and the attempt to make some money through Redis Labs is pretty recent in the scheme of things. If that helps with the continued development of Redis I have no problem with it.
- cletus 7y ago
  While I don't want to take anything away from memcached, I'm puzzled by the swipe at Redis. Whatever marketing you're referring to, Redis supports some pretty useful data structures that make some pretty common query patterns trivial (eg top-N queries) and does it with a small, elegant core of some of the most readable code ever produced. Memcache OTOH is a fairly simple KV cache. It seems to be pretty decent. Neither of these supplants the other however. Hence my puzzlement. It just seems out of place, unnecessary and doesn't really add anything.
philippz 7y ago
Been using memcached for >7 years now. Just thank you! It's been adding great benefits to almost all of my projects.
darknrgy 7y ago
Memcached is state of the art for caching and most of the big players on the internet use it. Nice work, dormando.
ksec 7y ago
It is interesting to see how Memcached and Redis, what was once competing in the same space, diverge in development. Redis continue to add features on top of features, it isn't bloat by any means, but it is still a lot more than say Redis 2.x. While Memcached stayed pretty much the same as memcahched 1.4, which was released over a decade ago. Optimising every single bit out of it and allow cache to be extended to SSD. Boring Tech is Good. Memcached should be added to that list. Edit: Release Note Mentioned Netflix, I wonder what they use it for.
- threeseed 7y ago
  > Boring Tech is Good It's fine if all you are doing is boring things. But often there are use cases where a simple key/value store just isn't going to cut it. And I for one am glad to have Redis as a solid, well-engineered alternative.
  - gdy 7y ago
    You can do boring things with cool tech and cool things with boring tech. I'm not sure if they even correlate.
- Rafuino 7y ago
  Regarding Netflix, here's a big usage of theirs, which is the Extstore (external flash storage) the Github page refers to. They probably use it beyond this, but it's a good technical article to sink your teeth into. https://netflixtechblog.com/evolution-of-application-data-caching-from-ram-to-ssd-a33d6fa7a690 https://netflixtechblog.com/evolution-of-application-data-ca...
subsaharancoder 7y ago
Snapchat is one of the largest if not the largest user of memcached on GCP, a few years ago they were +100TB and growing like a weed!!
- Tostino 7y ago
  That seems horrible.
kator 7y ago
What I like about Memcached is it's just that, it works, written in C, oh and my goodness uses Perl tests! It's not re-written every other year in the latest hot language. It chugs along, without embarrassment, doing one thing amazingly well over 17 years...
- bigiain 7y ago
  It used to be written in Perl, back in the day... I suspect some of those tests are quite likely to be from back then. I smiled when I read the title, thinking "How cool! Something I was using at a job I left ~16 years ago has _finally_ made it to version 1.6!" :-) (I suspect the old LiveJournal Perl code probably didn't have it's version numbers carry over into this somewhat more modern version...)
- threeseed 7y ago
  > It's not re-written every other year in the latest hot language What is with all of the weird sniping in this thread ? There are legitimate reasons why people move to Rust and similar languages e.g. security, maintainability.
- manigandham 7y ago
  > "It's not re-written every other year in the latest hot language." What project does this?
- qatanah 7y ago
  Redis tests are written in tcl! both perl and tcl are awesome!
pachico 7y ago
This is not a rhetorical question, I'm asking our of curiosity/ignorance. If you were about to start a new project, for what reasons would you choose Memcached over Redis? Or probably better said, for which use cases?
- rawoke083600 7y ago
  I grew up with memcached and it certainly has way less features than redis, but nothing is simpler. For linux+php anyway. From setup to usage.
- teej 7y ago
  Redis. Back in the day when I ran memcached with the rest of the cool kids, I always wanted something like SADD and LPUSH. Redis is a realization of that vision and using it is always a delight to me.
- mrweasel 7y ago
  I don't really see those two projects as solving the same problems. Memcache is for caching only, it's very good at it, there is basically no administrative overhead and people (i.e. other developers) know that if you stuff data into memcache, you should be prepared to lose it. Redis is, to me at least, much more akin to "shared data structures as a service". Sure it's also a key value store, where I could cache data, but I could do that with MySQL as well. I look to Redis when I need to have data shared between components, and sometimes a key value store happens to be the best solution, other times it may be pub/sub.
  - pachico 7y ago
    Thanks for your answer. I can hardly see Redis as a hard to admin software - vanilla installation should work in most (if not all) scenarios, honestly. This is why I'm asking because, giving that Redis has a great number of extra features and superb performance, it would be hard to justify, at least for me, to choose Memcached. Worst case scenario, you don't use all those extra features. Or am I missing anything?
    - 7y ago

110 points by nrr 7y ago

I have a trading bot that consolidates a lot of price information in order to make decisions. I'm currently using Redis due to data structures (lists and scores) and Pub/Sub. I wonder if memcached would give me performance benefits and lower my CPU costs.

manigandham 7y ago
CPU is cheap and memcached has no advanced data structures. If you're using them then you'll probably have to reimplement everything yourself in memcached with less efficiency.
- henriquez 7y ago
  You can always just JSON-encode whatever you’re putting in there and decode on the way out. Though I couldn’t imagine using it for pub/sub like the parent commenter is doing with redis.
  - manigandham 7y ago
    Which is inefficient and pointless compared to just using Redis with data structures that you can manipulate directly.
dormando 7y ago
Mind giving some examples in more detail about how you make use of lists/scores?
- haolez 7y ago
  Pub/Sub is easily replaceable. I use lists and scores to store orders from orderbooks, where the prices are the scores. Then I can order by price. It's simple, but I have like 400 updates per second.
jasonwatkinspdx 7y ago
You'll get a bunch of garbage answers here to be frank, from people who never put more than trivial load through both. Memcache blows redis out of the water in terms of raw throughput and latency, like it's not even remotely close on modern lots of cores and big memory machines. Redis has a more feature rich api, but has fundamentally hobbled itself on a root hash table and concurrency implementation that is circa 1990. Give memcached a try, even if it means you implement some stuff you currently defer to redis ops. Measure it. Look at the code bases. Make up your own mind about performance and which actually makes your code simpler.
- dijit 7y ago
  http://antirez.com/news/94 http://antirez.com/news/94 TL;DR: Redis is just doing different things, and the slab allocator is pretty efficient in regards to performance but not memory usage.
  - dormando 7y ago
    Fwiw this post comparison is pretty old. The memory usage should be the same or better these days (automatic slab rebalancing, LRU expired item crawler, chunked items, etc).
- haolez 7y ago
  Thanks. I'll make sure to create a PoC and post back here.

dleslie 7y ago

This was the first non-sql datastorage I used for online services; a lifetime ago. Glad to see that it is still maintained.

qeternity 7y ago
I wouldn't be surprised if there is more data cached by memcached than redis with all the massive players using. Hardly a dead project.
henriquez 7y ago
I still use Memcache in a handful of old PHP projects I’ve been keeping on life support since the mid 2000s. I’m also glad it’s being maintained! It’s perfectly fine at what it was designed to do.

coleifer 7y ago

Memcached doesn't pander or resort to marketing to achieve a userbase (compare to redis/labs). It's a great piece of software engineering and I absolutely respect that there's no gimmick, no bullshit. Just a lot of hard work given away for free. Great work!

dormando 7y ago
Thanks!
- cgidriver 7y ago
  Thanks for all your efforts! It is indeed a very useful piece of software. Easy to install and use, well documented, and very helpful. Is there documentation to run it distributed somewhere you can point me to?
vosper 7y ago
> Memcached doesn't pander or resort to marketing to achieve a userbase (compare to redis/labs) I think this is pretty unfair to Redis. Redis has a lot more features (tell me which ones you think are pandering, and to whom?), is free for the vast majority of what people want to use it for, and the attempt to make some money through Redis Labs is pretty recent in the scheme of things. If that helps with the continued development of Redis I have no problem with it.
cletus 7y ago
While I don't want to take anything away from memcached, I'm puzzled by the swipe at Redis. Whatever marketing you're referring to, Redis supports some pretty useful data structures that make some pretty common query patterns trivial (eg top-N queries) and does it with a small, elegant core of some of the most readable code ever produced. Memcache OTOH is a fairly simple KV cache. It seems to be pretty decent. Neither of these supplants the other however. Hence my puzzlement. It just seems out of place, unnecessary and doesn't really add anything.

philippz 7y ago

Been using memcached for >7 years now. Just thank you! It's been adding great benefits to almost all of my projects.

darknrgy 7y ago

Memcached is state of the art for caching and most of the big players on the internet use it. Nice work, dormando.

ksec 7y ago

It is interesting to see how Memcached and Redis, what was once competing in the same space, diverge in development. Redis continue to add features on top of features, it isn't bloat by any means, but it is still a lot more than say Redis 2.x. While Memcached stayed pretty much the same as memcahched 1.4, which was released over a decade ago. Optimising every single bit out of it and allow cache to be extended to SSD. Boring Tech is Good. Memcached should be added to that list. Edit: Release Note Mentioned Netflix, I wonder what they use it for.

threeseed 7y ago
> Boring Tech is Good It's fine if all you are doing is boring things. But often there are use cases where a simple key/value store just isn't going to cut it. And I for one am glad to have Redis as a solid, well-engineered alternative.
- gdy 7y ago
  You can do boring things with cool tech and cool things with boring tech. I'm not sure if they even correlate.
Rafuino 7y ago
Regarding Netflix, here's a big usage of theirs, which is the Extstore (external flash storage) the Github page refers to. They probably use it beyond this, but it's a good technical article to sink your teeth into. https://netflixtechblog.com/evolution-of-application-data-caching-from-ram-to-ssd-a33d6fa7a690 https://netflixtechblog.com/evolution-of-application-data-ca...

subsaharancoder 7y ago

Snapchat is one of the largest if not the largest user of memcached on GCP, a few years ago they were +100TB and growing like a weed!!

Tostino 7y ago
That seems horrible.

kator 7y ago

What I like about Memcached is it's just that, it works, written in C, oh and my goodness uses Perl tests! It's not re-written every other year in the latest hot language. It chugs along, without embarrassment, doing one thing amazingly well over 17 years...

bigiain 7y ago
It used to be written in Perl, back in the day... I suspect some of those tests are quite likely to be from back then. I smiled when I read the title, thinking "How cool! Something I was using at a job I left ~16 years ago has _finally_ made it to version 1.6!" :-) (I suspect the old LiveJournal Perl code probably didn't have it's version numbers carry over into this somewhat more modern version...)
threeseed 7y ago
> It's not re-written every other year in the latest hot language What is with all of the weird sniping in this thread ? There are legitimate reasons why people move to Rust and similar languages e.g. security, maintainability.
manigandham 7y ago
> "It's not re-written every other year in the latest hot language." What project does this?
qatanah 7y ago
Redis tests are written in tcl! both perl and tcl are awesome!

pachico 7y ago

This is not a rhetorical question, I'm asking our of curiosity/ignorance. If you were about to start a new project, for what reasons would you choose Memcached over Redis? Or probably better said, for which use cases?

rawoke083600 7y ago
I grew up with memcached and it certainly has way less features than redis, but nothing is simpler. For linux+php anyway. From setup to usage.
teej 7y ago
Redis. Back in the day when I ran memcached with the rest of the cool kids, I always wanted something like SADD and LPUSH. Redis is a realization of that vision and using it is always a delight to me.
mrweasel 7y ago
I don't really see those two projects as solving the same problems. Memcache is for caching only, it's very good at it, there is basically no administrative overhead and people (i.e. other developers) know that if you stuff data into memcache, you should be prepared to lose it. Redis is, to me at least, much more akin to "shared data structures as a service". Sure it's also a key value store, where I could cache data, but I could do that with MySQL as well. I look to Redis when I need to have data shared between components, and sometimes a key value store happens to be the best solution, other times it may be pub/sub.
- pachico 7y ago
  Thanks for your answer. I can hardly see Redis as a hard to admin software - vanilla installation should work in most (if not all) scenarios, honestly. This is why I'm asking because, giving that Redis has a great number of extra features and superb performance, it would be hard to justify, at least for me, to choose Memcached. Worst case scenario, you don't use all those extra features. Or am I missing anything?
  - 7y ago