5 ms·

An update on the Memcached/Redis benchmark

33 points by spahl 16y ago

wglb 16y ago
The pursuit of benchmarks is fraught with peril. So it is nice to see antirez taking care to evaluate and thoroughly analyze where the differences are between two benchmarks. One of the conclusions that I take away from this is that threading is not a net win in the cloud. CPUs are so cheap that (to a first-order approximation) spending development time on threaded programs takes development time away from developing features.
vosper 16y ago
I'm enjoying following this back-and-forth between memcached and Redis; I learn something from each post. And it's nice to see it being conducted in such a gentlemanly fashion!
jacquesm 16y ago
key insight here: > in a future of cloud computing I want to consider every single core as a computer itself. And that leads directly to a shared nothing architecture, which I think is definitely an advantage. It's KISS all over, multi-threading is often not the right decision. It leads to more complex (and sometimes very hard to debug) software that will have only marginal gain (or even a drop as in this case) over running multiple well behaved instances of the same process.
- ergo98 16y ago
  key insight here: > in a future of cloud computing I want to consider every single core as a computer itself. I didn't interpret that as an insight more than a rather weak attempt at squeezing some lemonade out of lemons. I also don't see how creating massive sharding on a single machine in any way corresponds with KISS (the newest processors have from 8 to 12 cores. 24 cores and up is coming in the very near future. So on my 48-core 4-processor relatively low end machine I'm to shard Redis 48 ways? I'm to run 48 instances of Linux? I'm to endure a devastation of cache coherency?)
  - jacquesm 16y ago
    Cloud computing was mentioned, we weren't looking at your specific machine but at a computing environment where the commodities are cycles, cores, bandwidth and storage. In such an environment firing up another instance of your process to match the number of free cores available should be done with as little horizontal communication between processes as possible.
  - antirez 16y ago
    Ergo we plan to add options so that Redis can do this automatically for you. You'll be able to say, start N instances. And when clustering will be available you can configure all this instances with a single common configuration. For now this should be done manually, but you have 4 or 8 cores so it's not a big problem IMHO. The idea is that there is no difference between the instances you have among different servers, and the ones you have locally. Given that every instance can hold a set of unrelated keys, why centralizing it in some way if completely separated threads can lead to max performances, and simpler code? Note that 48 independent threads or 48 processes are actually the same thing more or less, but nothing prevents us from starting N threads at startup if needed for some reason in the future, each with completely separated state. I don't understand your parallel about starting N operating systems btw... Anyway, this is not a problem just with Redis. Programming languages like Ruby are a mess with the usual conception of threads. And most Ruby programmers also think far too high level to really deal with a low level stuff such as synchronization. A shared nothing model with message passing, and one interpreter per thread, could lead to the maximum benefit while retaining the more abstract interface to the programmer. Believe it or not the time for old-fashion multi thread programming is gone (I hope).
    - jacquesm 16y ago
      Or even change the number of processes on demand.
    - shin_lao 16y ago
      48 independent threads and 48 processes can be very different on some operating systems. A process is more heavyweight. You will - amongst other things - increase the charge in terms of handles. For example, 48 processes maintaining the same network connections is a waste of resources. I'm afraid you might hit a wall with your single threaded approach. This is definitely not the road we have chosen (but we wish the best of luck nonetheless! :) ).

33 points by spahl 16y ago

wglb 16y ago

The pursuit of benchmarks is fraught with peril. So it is nice to see antirez taking care to evaluate and thoroughly analyze where the differences are between two benchmarks. One of the conclusions that I take away from this is that threading is not a net win in the cloud. CPUs are so cheap that (to a first-order approximation) spending development time on threaded programs takes development time away from developing features.

vosper 16y ago

I'm enjoying following this back-and-forth between memcached and Redis; I learn something from each post. And it's nice to see it being conducted in such a gentlemanly fashion!

jacquesm 16y ago

key insight here: > in a future of cloud computing I want to consider every single core as a computer itself. And that leads directly to a shared nothing architecture, which I think is definitely an advantage. It's KISS all over, multi-threading is often not the right decision. It leads to more complex (and sometimes very hard to debug) software that will have only marginal gain (or even a drop as in this case) over running multiple well behaved instances of the same process.

ergo98 16y ago
key insight here: > in a future of cloud computing I want to consider every single core as a computer itself. I didn't interpret that as an insight more than a rather weak attempt at squeezing some lemonade out of lemons. I also don't see how creating massive sharding on a single machine in any way corresponds with KISS (the newest processors have from 8 to 12 cores. 24 cores and up is coming in the very near future. So on my 48-core 4-processor relatively low end machine I'm to shard Redis 48 ways? I'm to run 48 instances of Linux? I'm to endure a devastation of cache coherency?)
- jacquesm 16y ago
  Cloud computing was mentioned, we weren't looking at your specific machine but at a computing environment where the commodities are cycles, cores, bandwidth and storage. In such an environment firing up another instance of your process to match the number of free cores available should be done with as little horizontal communication between processes as possible.
- antirez 16y ago
  Ergo we plan to add options so that Redis can do this automatically for you. You'll be able to say, start N instances. And when clustering will be available you can configure all this instances with a single common configuration. For now this should be done manually, but you have 4 or 8 cores so it's not a big problem IMHO. The idea is that there is no difference between the instances you have among different servers, and the ones you have locally. Given that every instance can hold a set of unrelated keys, why centralizing it in some way if completely separated threads can lead to max performances, and simpler code? Note that 48 independent threads or 48 processes are actually the same thing more or less, but nothing prevents us from starting N threads at startup if needed for some reason in the future, each with completely separated state. I don't understand your parallel about starting N operating systems btw... Anyway, this is not a problem just with Redis. Programming languages like Ruby are a mess with the usual conception of threads. And most Ruby programmers also think far too high level to really deal with a low level stuff such as synchronization. A shared nothing model with message passing, and one interpreter per thread, could lead to the maximum benefit while retaining the more abstract interface to the programmer. Believe it or not the time for old-fashion multi thread programming is gone (I hope).
  - jacquesm 16y ago
    Or even change the number of processes on demand.
  - shin_lao 16y ago
    48 independent threads and 48 processes can be very different on some operating systems. A process is more heavyweight. You will - amongst other things - increase the charge in terms of handles. For example, 48 processes maintaining the same network connections is a waste of resources. I'm afraid you might hit a wall with your single threaded approach. This is definitely not the road we have chosen (but we wish the best of luck nonetheless! :) ).