4 ms·
FYI it is a little suspect to write your own testing framework and then claim a ridiculous performance gain over something like Redis. There are already widely
by mike_d 3y ago
FYI it is a little suspect to write your own testing framework and then claim a ridiculous performance gain over something like Redis. There are already widely accepted testing tools [1] you could have used.
It might make more sense to write a harness that allows your software to be used with standard Redis wire protocol so it can be properly benchmarked and compared against the dozens of existing solutions in the space.
Also it seems like you accidentally discovered the Latency Numbers Everyone Should Know [2]. Local operations are faster than network operations.
1. https://redis.io/docs/management/optimization/benchmarks/ https://redis.io/docs/management/optimization/benchmarks/
2. https://static.googleusercontent.com/media/sre.google/en//static/pdf/rule-of-thumb-latency-numbers-letter.pdf https://static.googleusercontent.com/media/sre.google/en//st...
- pushkarg 3y agoExactly! Accessing data in RAM will be orders of magnitude faster than over the network (maybe this is a good sanity check of our benchmark numbers). The core principle behind IKV is that it allows you to access (large) data-sets without network calls. Most DB tooling out there only works for a client-server model. And implementing the Redis protocol would imply changing our architecture significantly and negatively affect performance (ex. a producer-consumer queue to serve requests, ser-deserialization costs).
- mike_d 3y ago> implementing the Redis protocol would imply changing our architecture significantly and negatively affect performance Yes, you’d be doing a fair and honest benchmark if you want to compare yourself to Redis.
- pushkarg 3y agoI just explained how implementing the Redis protocol will inherit inefficiencies of Redis in IKV - so I don't understand how the comparison will be fair or honest. Hope this addresses your original question about why we wrote a custom benchmarking client. https://github.com/inlinedio/ikv-java-client/blob/master/src/main/java/io/inlined/benchmarks/runner/MultiThreadedRunner.java https://github.com/inlinedio/ikv-java-client/blob/master/src... It is quite simple and is available here. There is nothing malicious in there to make IKV appear faster. Although ff you do see a bug, I am happy to fix and republish results.