4 ms·

Yes, I can confirm that Memcached can reach similar performance as DF. However, one of the goals of DF was to combine the performance of Memcached with versatil

Yes, I can confirm that Memcached can reach similar performance as DF. However, one of the goals of DF was to combine the performance of Memcached with versatility of Redis. I implemented an engine that provides atomicity guarantees for all its operations plus transparent snapshotting under write-heavy traffic and all this without reducing the performance compared to memcached. Having said that, DF also has a novel caching algorithm that should provide better hit rate with less memory consumption.

xiphias2 4y ago
Do the benchmarks stress test those atomicity guarantees? Get/set operations look like they don't need it.
- romange 4y ago
  You are correct - GET/SET do not require any locking as long as they do not contend and they do not in those benchmarks. You are right that for MSET/MGET you will see lower numbers. But still it will be much higher than with REDIS. This is our initial release and we just did not have resources to showcase everything under different scenarios. Having said that, if you open an issue with a suggestion of a benchmark that you would like to see I will try to run soon...
  - xiphias2 4y ago
    No worries, I trust you to do the right thing, just measuring something that Memcached can do fast anyways can be a misleading benchmark.
- anitil 4y ago
  Do you know what tooling is available for testing atomicity guarantees?
  - romange 4y ago
    The best tool out there is probably jepsen.io but it requires domain knowledge to properly operate it. The guy behind it is a beast! Anyway, I wrote lots of unit tests to cover those atomicity issues. You can also write a custom python/nodejs/golang/... scripts that simultenusly write and read from the same multiple keys in such way that some invariant is preserved. For example, "mset x {i}, y {i}" for random `i` and in parallel do "mget x y" and to check that the response returns same values. You can also test this for other families using transactions like "MULTI; lpush x ${foo}; lpush y ${foo}; EXEC" .. and then similarly test that x and y have exactly the same lists.
    - anitil 4y ago
      Ah thanks I had heard of and then forgotten about jepsen. I had considered what you described but I thought there might be a more targeted approach where you can specify your invariants (like "x can only be in one of two states" or "pushing alway increases the length of a list and popping always reduces it if it's non-empty") and get the tool to try to break them. I wonder if there could be a tool that instruments in the same way that afl does to try to detect races or inconsistent states.

xiphias2 4y ago

Do the benchmarks stress test those atomicity guarantees? Get/set operations look like they don't need it.

romange 4y ago
You are correct - GET/SET do not require any locking as long as they do not contend and they do not in those benchmarks. You are right that for MSET/MGET you will see lower numbers. But still it will be much higher than with REDIS. This is our initial release and we just did not have resources to showcase everything under different scenarios. Having said that, if you open an issue with a suggestion of a benchmark that you would like to see I will try to run soon...
- xiphias2 4y ago
  No worries, I trust you to do the right thing, just measuring something that Memcached can do fast anyways can be a misleading benchmark.
anitil 4y ago
Do you know what tooling is available for testing atomicity guarantees?
- romange 4y ago
  The best tool out there is probably jepsen.io but it requires domain knowledge to properly operate it. The guy behind it is a beast! Anyway, I wrote lots of unit tests to cover those atomicity issues. You can also write a custom python/nodejs/golang/... scripts that simultenusly write and read from the same multiple keys in such way that some invariant is preserved. For example, "mset x {i}, y {i}" for random `i` and in parallel do "mget x y" and to check that the response returns same values. You can also test this for other families using transactions like "MULTI; lpush x ${foo}; lpush y ${foo}; EXEC" .. and then similarly test that x and y have exactly the same lists.
  - anitil 4y ago
    Ah thanks I had heard of and then forgotten about jepsen. I had considered what you described but I thought there might be a more targeted approach where you can specify your invariants (like "x can only be in one of two states" or "pushing alway increases the length of a list and popping always reduces it if it's non-empty") and get the tool to try to break them. I wonder if there could be a tool that instruments in the same way that afl does to try to detect races or inconsistent states.