8 ms·

Drop-In Replacement for Memcached

143 points by mfrw 5y ago

lumost 5y ago
Biggest value prop seems to be that it's a single binary built off cargo, and that it's rust based. I wouldn't be surprised to see “single binary” become a selling point for container based shops even if all else is equal.
- galkk 5y ago
  Soon we'll see entire docker compose stack packed into single binary and everything will go back again
  - qaq 5y ago
    It's pretty convenient options for local dev. I am working on a project that has 7 services but they can be bundled into a single binary that has all 7 and NATS packaged in (plus you can spin up multiple instances of the services). Very convenient and builds in a few seconds.
  - aledalgrande 5y ago
    And after a few years, "modular architecture with dynamically linked libraries! lean binaries!"
- evanelias 5y ago
  AFAIK, memcached's only mandatory external dependency is libevent. And you can statically link that if you want.
- mrweasel 5y ago
  Unless you're doing something very special, I'd guess that most container based solution will just get the memcached Docker image on Docker Hub. There's very little to be gained by building your own memcached image, with a "single binary" alternative.
nkuttler 5y ago
Hm, would be nice if they had a comparison. More than just "it's Rust".
- deleted 5y ago
  [deleted]
montroser 5y ago
Cool, but what problem is this meant to solve? How does it compare to memcached in terms of performance?
- mfrw 5y ago
  I am not among the ones who would blindly say 'rewrite it in rust', but given what happened with log4j, I would much rather choose a safe implementation given the memory safety provided by rust. This also does not mean rust might not have its fair share of bugs; IMHO the probability is a little less. Edit: I mentioned the log4j RCE as it was new & do not mean to imply that it would not have happened if we had a rust implementation. As the comments have mentioned, it would be the same in any language IMO
  - zamalek 5y ago
    > IMHO the probability is a little less. Exactly, Rust simply eliminates a certain class of bugs. You can still make some really major mistakes in the absence of these bugs, e.g. "goto fail" could have easily been written in Rust.
    - ordiel 5y ago
      Most certainly opening the door to a diffetent kind of bugs. This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves, and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use. Remember heartbleed?...
      - staticassertion 5y ago
        > Most certainly opening the door to a diffetent kind of bugs. No? There's no law of conservation of bug class or something, you can eliminate bug classes without introducing a new one. > This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves, No it doesn't. > and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use. I don't think this has held true historically, and log4j is a good example of this (though there are many others). > Remember heartbleed?... A weird point to bring up since heartbleed was an out of bounds memory read.
aftbit 5y ago
What can this do that memcached can't? We use memcached heavily in PROD but I would be hard pressed to replace it with an upstart competitor just because its is possible.
- buro9 5y ago
  Memcache is a BSD licence. This is AGPL or commercial. The biggest difference I could find was not in the capability of it as a drop-in replacement but in the change of licence as any changes you might have to fit your needs now has different terms. I guess the bet here is predicated on that... are companies running versions of memcached that are much improved and they're not sharing those improvements? Could this be a way to make those improvements public by this replacing memcached over time? I'm not sure how they could do a commercial licence once they've accepted contributions, if they don't have a CLA granting the right over the contributions. But IANAL, etc.
  - erk__ 5y ago
    They do indeed have a CLA: https://cla-assistant.io/memc-rs/memc-rs https://cla-assistant.io/memc-rs/memc-rs
    - buro9 5y ago
      Ah, awesome. I did not see this mentioned in the contributing.md file (it may have been there, I just didn't notice it).
  - CodeWriter23 5y ago
    > Could this be a way to make those improvements public by this replacing memcached over time? Those who wish to share their contributions will, regardless of license. Those who do not, will stick to code with BSD-style licenses.
  - adrr 5y ago
    Doesn’t matter the license. If I modified this code and didn’t distribute the code, I have 0 obligation to publish/share my changes. These are copyright licenses and license agreements. Distribution is the trigger for copyright licenses.
welder 5y ago
Stop using RAM for caching, SSDs are just as fast now with LevelDB/RocksDB and can store a magnitude more data. Look into SSDB [1] instead of Memcached. I know it's Redis, but it's basically the same as Memcached if you only use the key-value set/get commands. See this blog post [2] and this Gophercon 2021 talk [3] for examples of it's use in production. [1] https://github.com/ideawu/ssdb https://github.com/ideawu/ssdb [2] https://wakatime.com/blog/45-using-a-diskbased-redis-clone-to-reduce-aws-s3-bill https://wakatime.com/blog/45-using-a-diskbased-redis-clone-t... [3] https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s
- ec109685 5y ago
  SSDB is faster even if data size is >> ram size? Otherwise, you’re going to be adding equivalent memory to get to equivalent performance between memcached and ssdb.
- jeffbee 5y ago
  I mean, hard no on both claims. A memory cache (even memcached, which might not be state of the art) stomps all over the performance claims of SSDB. Memcached can do ~250kqps on one core, roughly depending of exactly what kind of cores are in question, and can typically turn around a request in < 50µs. SSDB is only "just as fast" if the rest of your stack is so insensitive to performance that you can't measure an order of magnitude difference in service latency.
  - scrubs 5y ago
    agree - most sstables use memory looking on disk if needed. ssds do not have the bandwidth or low latency RAM has apart from the additional I/O path to get it onto or out of disk. Now, not to say memory+ssd can't rock ... but ssd!=DDRAM.
  - tayo42 5y ago
    > which might not be state of the art Sorry this stuck out to me. what would be state of the art in the in memory kv store world then?
awestroke 5y ago
This seems to be a concurrent hashmap (dashmap[1]) with a server speaking the memcached protocol. I know both memcached and redis use sophisticated strategies to reduce allocation calls and memory fragmentation, but this seems to do nothing like that. [1]: https://docs.rs/dashmap/latest/dashmap/ https://docs.rs/dashmap/latest/dashmap/
- CodesInChaos 5y ago
  > use sophisticated strategies to reduce allocation calls and memory fragmentation Both of these are things a high quality allocator (jemalloc in this case) has to to optimize for as well (in this context "allocation calls" corresponds to syscalls). For example they also allocate different sizes from different buckets. Applications do have some abilities an allocator does not have, for example: * It can move allocations, since it can invalidate pointers. This allows defragmentation/compaction to happen * It can try to predict how long lived an allocation will be But it's not clear to me if any of these are important for performance.
- rurban 5y ago
  Memcache uses an outdated and slow hash and hashtable. This is using jemalloc and a concurrent hashtable. Seems to be miles better.
  - Lhiw 5y ago
    Not this again... https://news.ycombinator.com/item?id=28833202 https://news.ycombinator.com/item?id=28833202
  - tinktank 5y ago
    > Memcache uses an outdated and slow hash and hashtable. No it doesn't. A cursory scan through the code will tell you as much.
    - rurban 5y ago
      Exactly why I told so. I even started to improve it some years ago, but got sidetracked. The only nice thing about the code are signals on htable updates. But a proper, fast concurrent hashtable would work wonders.
nvr219 5y ago
RIP LiveJournal
maxpert 5y ago
Sometime back I did a Golang based memcache server implementation with BoltDB (experimented with Pebble as well). I dropped that implementation out of fear that nobody will be interested in using it, have same question here I guess. Why would I drop the existing battle tested solution for something that might still have non-memory bugs.
keyle 5y ago
s/Drop-In Replacement/Rust partial reimplementation/
- reyqn 5y ago
  Reading their github I was under the impression that it was an actual drop in replacement, their capability status passed all tests from memcapable. Am I missing something?
  - tayo42 5y ago
    it wont perform as well, memcached does a lot of its own memory management. this defers all the storage to a library. the eviction policy is also not the same. random eviction might catch some people off guard if your expecting what memcached does. it doesnt seem to support the ascii protocol either. didnt see anything similar to disk based storage or persistent storage memcached has now
nodesocket 5y ago
Does memc.rs have the ability to do replication (master/slave)?
NaturalPhallacy 5y ago
Ok. Why?
- lopatin 5y ago
  Because Memcached wasn't written by the OP, but this library was.
maxmcd 5y ago
memc has a measuring performance section: https://github.com/memc-rs/memc-rs#measuring-performance https://github.com/memc-rs/memc-rs#measuring-performance As an arbitrary example, on my machine if I run memtier_benchmark like so: memtier_benchmark --port=11211 --protocol=memcache_binary \ -t 4 --ratio=1:1 --pipeline=9 -c 16 \ -d 100 --key-pattern=S:S --key-minimum=1 \ --key-maximum=1000000 -n allkeys I get the following results: memc: 63999483 ops, 587092 ops/sec, 51.39MB/sec, avg 0.82 msec latency memcache: 63999936 ops, 2391777 ops/sec, 209.34MB/sec, avg 0.24 msec latency memtier_benchmark also prints out more detailed stats, but I couldn't get those to work for memc. Not sure what's going on there. If we set the set/get ratio to 1:10 (maybe more realistic), they get much closer: memc: 6399480 ops, 639989 ops/sec, 28.45MB/sec, avg 0.67 msec latency memcache: 6399623 ops, 1291566 ops/sec, 57.41MB/sec, avg 0.40 msec latency
- injinj 5y ago
  A wireshark capture shows that memcrsd is breaking up the response packets, not optimizing network, sending a lot more packets than memcached. With the 100 byte size, memcached is packing 10 PDUs per packet and memcrsd is doing 1. Using memtier_benchmark --pipeline=1 with "memcached -t 1" and "memcrsd -r 1" shows almost identical results (141943 vs 136550) on my machine (keys 1 -> 100000). The network is where most of the CPU is spent in both memcached and memcrsd cases. I'm slightly impressed with memcrsd/rust scaling, 32 runtimes with 16 memtier_benchark threads with pipeline=9 can produce 4 million ops/sec. They need to fix the networking parts. I think most will use horizontal scaling anyway, and simply shard the memcached's and skip the threading concurrency. Multiple memcached's with client side sharding is hard to beat.
harryf 5y ago
I don't get it. Just starting with the three main benefits listed on the website, none of these are compelling... > Safe: Implemented in Rust programming language and using Tokio runtime ensures memory and thread safety. Does memcached have issues with memory and thread safety? I've never heard of them if it does. > Fast: Can process thousands of requests with minimal overhead. For a network cache you want _predictable_ performance more than "fast" performance, so that you're able to predict how a system will scale. "Fast" is among the top priorities for choosing a network cache implementation. > Compatible: memc.rs is 100% compatible with memcached binary protocol. No need to rewrite your code. Just start a server and use your favorite language/framework. What about tools for monitoring memcached in production? Does it support consistent hashing across a pool memcached servers? Ultimately: why?
- akx 5y ago
  Well yes, memcached has apparently had its share of memory-corruption vulnerabilities. https://www.cvedetails.com/vulnerability-list/vendor_id-12993/product_id-26610/Memcached-Memcached.html https://www.cvedetails.com/vulnerability-list/vendor_id-1299...
tommica 5y ago
It's great that they made it, it's always good to have alternatives, and when they are made this well, it is just good for the ecosystem.

143 points by mfrw 5y ago

Biggest value prop seems to be that it's a single binary built off cargo, and that it's rust based. I wouldn't be surprised to see “single binary” become a selling point for container based shops even if all else is equal.

galkk 5y ago
Soon we'll see entire docker compose stack packed into single binary and everything will go back again
- qaq 5y ago
  It's pretty convenient options for local dev. I am working on a project that has 7 services but they can be bundled into a single binary that has all 7 and NATS packaged in (plus you can spin up multiple instances of the services). Very convenient and builds in a few seconds.
- aledalgrande 5y ago
  And after a few years, "modular architecture with dynamically linked libraries! lean binaries!"
evanelias 5y ago
AFAIK, memcached's only mandatory external dependency is libevent. And you can statically link that if you want.
mrweasel 5y ago
Unless you're doing something very special, I'd guess that most container based solution will just get the memcached Docker image on Docker Hub. There's very little to be gained by building your own memcached image, with a "single binary" alternative.

nkuttler 5y ago

Hm, would be nice if they had a comparison. More than just "it's Rust".

deleted 5y ago
[deleted]

montroser 5y ago

Cool, but what problem is this meant to solve? How does it compare to memcached in terms of performance?

mfrw 5y ago
I am not among the ones who would blindly say 'rewrite it in rust', but given what happened with log4j, I would much rather choose a safe implementation given the memory safety provided by rust. This also does not mean rust might not have its fair share of bugs; IMHO the probability is a little less. Edit: I mentioned the log4j RCE as it was new & do not mean to imply that it would not have happened if we had a rust implementation. As the comments have mentioned, it would be the same in any language IMO
- zamalek 5y ago
  > IMHO the probability is a little less. Exactly, Rust simply eliminates a certain class of bugs. You can still make some really major mistakes in the absence of these bugs, e.g. "goto fail" could have easily been written in Rust.
  - ordiel 5y ago
    Most certainly opening the door to a diffetent kind of bugs. This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves, and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use. Remember heartbleed?...
    - staticassertion 5y ago
      > Most certainly opening the door to a diffetent kind of bugs. No? There's no law of conservation of bug class or something, you can eliminate bug classes without introducing a new one. > This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves, No it doesn't. > and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use. I don't think this has held true historically, and log4j is a good example of this (though there are many others). > Remember heartbleed?... A weird point to bring up since heartbleed was an out of bounds memory read.

aftbit 5y ago

What can this do that memcached can't? We use memcached heavily in PROD but I would be hard pressed to replace it with an upstart competitor just because its is possible.

buro9 5y ago
Memcache is a BSD licence. This is AGPL or commercial. The biggest difference I could find was not in the capability of it as a drop-in replacement but in the change of licence as any changes you might have to fit your needs now has different terms. I guess the bet here is predicated on that... are companies running versions of memcached that are much improved and they're not sharing those improvements? Could this be a way to make those improvements public by this replacing memcached over time? I'm not sure how they could do a commercial licence once they've accepted contributions, if they don't have a CLA granting the right over the contributions. But IANAL, etc.
- erk__ 5y ago
  They do indeed have a CLA: https://cla-assistant.io/memc-rs/memc-rs https://cla-assistant.io/memc-rs/memc-rs
  - buro9 5y ago
    Ah, awesome. I did not see this mentioned in the contributing.md file (it may have been there, I just didn't notice it).
- CodeWriter23 5y ago
  > Could this be a way to make those improvements public by this replacing memcached over time? Those who wish to share their contributions will, regardless of license. Those who do not, will stick to code with BSD-style licenses.
- adrr 5y ago
  Doesn’t matter the license. If I modified this code and didn’t distribute the code, I have 0 obligation to publish/share my changes. These are copyright licenses and license agreements. Distribution is the trigger for copyright licenses.

welder 5y ago

Stop using RAM for caching, SSDs are just as fast now with LevelDB/RocksDB and can store a magnitude more data. Look into SSDB [1] instead of Memcached. I know it's Redis, but it's basically the same as Memcached if you only use the key-value set/get commands. See this blog post [2] and this Gophercon 2021 talk [3] for examples of it's use in production. [1] https://github.com/ideawu/ssdb https://github.com/ideawu/ssdb [2] https://wakatime.com/blog/45-using-a-diskbased-redis-clone-to-reduce-aws-s3-bill https://wakatime.com/blog/45-using-a-diskbased-redis-clone-t... [3] https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s

ec109685 5y ago
SSDB is faster even if data size is >> ram size? Otherwise, you’re going to be adding equivalent memory to get to equivalent performance between memcached and ssdb.
jeffbee 5y ago
I mean, hard no on both claims. A memory cache (even memcached, which might not be state of the art) stomps all over the performance claims of SSDB. Memcached can do ~250kqps on one core, roughly depending of exactly what kind of cores are in question, and can typically turn around a request in < 50µs. SSDB is only "just as fast" if the rest of your stack is so insensitive to performance that you can't measure an order of magnitude difference in service latency.
- scrubs 5y ago
  agree - most sstables use memory looking on disk if needed. ssds do not have the bandwidth or low latency RAM has apart from the additional I/O path to get it onto or out of disk. Now, not to say memory+ssd can't rock ... but ssd!=DDRAM.
- tayo42 5y ago
  > which might not be state of the art Sorry this stuck out to me. what would be state of the art in the in memory kv store world then?

awestroke 5y ago

This seems to be a concurrent hashmap (dashmap[1]) with a server speaking the memcached protocol. I know both memcached and redis use sophisticated strategies to reduce allocation calls and memory fragmentation, but this seems to do nothing like that. [1]: https://docs.rs/dashmap/latest/dashmap/ https://docs.rs/dashmap/latest/dashmap/

CodesInChaos 5y ago
> use sophisticated strategies to reduce allocation calls and memory fragmentation Both of these are things a high quality allocator (jemalloc in this case) has to to optimize for as well (in this context "allocation calls" corresponds to syscalls). For example they also allocate different sizes from different buckets. Applications do have some abilities an allocator does not have, for example: * It can move allocations, since it can invalidate pointers. This allows defragmentation/compaction to happen * It can try to predict how long lived an allocation will be But it's not clear to me if any of these are important for performance.
rurban 5y ago
Memcache uses an outdated and slow hash and hashtable. This is using jemalloc and a concurrent hashtable. Seems to be miles better.
- Lhiw 5y ago
  Not this again... https://news.ycombinator.com/item?id=28833202 https://news.ycombinator.com/item?id=28833202
- tinktank 5y ago
  > Memcache uses an outdated and slow hash and hashtable. No it doesn't. A cursory scan through the code will tell you as much.
  - rurban 5y ago
    Exactly why I told so. I even started to improve it some years ago, but got sidetracked. The only nice thing about the code are signals on htable updates. But a proper, fast concurrent hashtable would work wonders.

nvr219 5y ago

RIP LiveJournal

maxpert 5y ago

Sometime back I did a Golang based memcache server implementation with BoltDB (experimented with Pebble as well). I dropped that implementation out of fear that nobody will be interested in using it, have same question here I guess. Why would I drop the existing battle tested solution for something that might still have non-memory bugs.

keyle 5y ago

s/Drop-In Replacement/Rust partial reimplementation/

reyqn 5y ago
Reading their github I was under the impression that it was an actual drop in replacement, their capability status passed all tests from memcapable. Am I missing something?
- tayo42 5y ago
  it wont perform as well, memcached does a lot of its own memory management. this defers all the storage to a library. the eviction policy is also not the same. random eviction might catch some people off guard if your expecting what memcached does. it doesnt seem to support the ascii protocol either. didnt see anything similar to disk based storage or persistent storage memcached has now

nodesocket 5y ago

Does memc.rs have the ability to do replication (master/slave)?

NaturalPhallacy 5y ago

Ok. Why?

lopatin 5y ago
Because Memcached wasn't written by the OP, but this library was.

maxmcd 5y ago

memc has a measuring performance section: https://github.com/memc-rs/memc-rs#measuring-performance https://github.com/memc-rs/memc-rs#measuring-performance As an arbitrary example, on my machine if I run memtier_benchmark like so: memtier_benchmark --port=11211 --protocol=memcache_binary \ -t 4 --ratio=1:1 --pipeline=9 -c 16 \ -d 100 --key-pattern=S:S --key-minimum=1 \ --key-maximum=1000000 -n allkeys I get the following results: memc: 63999483 ops, 587092 ops/sec, 51.39MB/sec, avg 0.82 msec latency memcache: 63999936 ops, 2391777 ops/sec, 209.34MB/sec, avg 0.24 msec latency memtier_benchmark also prints out more detailed stats, but I couldn't get those to work for memc. Not sure what's going on there. If we set the set/get ratio to 1:10 (maybe more realistic), they get much closer: memc: 6399480 ops, 639989 ops/sec, 28.45MB/sec, avg 0.67 msec latency memcache: 6399623 ops, 1291566 ops/sec, 57.41MB/sec, avg 0.40 msec latency

injinj 5y ago
A wireshark capture shows that memcrsd is breaking up the response packets, not optimizing network, sending a lot more packets than memcached. With the 100 byte size, memcached is packing 10 PDUs per packet and memcrsd is doing 1. Using memtier_benchmark --pipeline=1 with "memcached -t 1" and "memcrsd -r 1" shows almost identical results (141943 vs 136550) on my machine (keys 1 -> 100000). The network is where most of the CPU is spent in both memcached and memcrsd cases. I'm slightly impressed with memcrsd/rust scaling, 32 runtimes with 16 memtier_benchark threads with pipeline=9 can produce 4 million ops/sec. They need to fix the networking parts. I think most will use horizontal scaling anyway, and simply shard the memcached's and skip the threading concurrency. Multiple memcached's with client side sharding is hard to beat.

harryf 5y ago

I don't get it. Just starting with the three main benefits listed on the website, none of these are compelling... > Safe: Implemented in Rust programming language and using Tokio runtime ensures memory and thread safety. Does memcached have issues with memory and thread safety? I've never heard of them if it does. > Fast: Can process thousands of requests with minimal overhead. For a network cache you want _predictable_ performance more than "fast" performance, so that you're able to predict how a system will scale. "Fast" is among the top priorities for choosing a network cache implementation. > Compatible: memc.rs is 100% compatible with memcached binary protocol. No need to rewrite your code. Just start a server and use your favorite language/framework. What about tools for monitoring memcached in production? Does it support consistent hashing across a pool memcached servers? Ultimately: why?

akx 5y ago
Well yes, memcached has apparently had its share of memory-corruption vulnerabilities. https://www.cvedetails.com/vulnerability-list/vendor_id-12993/product_id-26610/Memcached-Memcached.html https://www.cvedetails.com/vulnerability-list/vendor_id-1299...

tommica 5y ago

It's great that they made it, it's always good to have alternatives, and when they are made this well, it is just good for the ecosystem.