3 ms·

Ergo we plan to add options so that Redis can do this automatically for you. You'll be able to say, start N instances. And when clustering will be available you

by antirez 16y ago

Ergo we plan to add options so that Redis can do this automatically for you. You'll be able to say, start N instances. And when clustering will be available you can configure all this instances with a single common configuration. For now this should be done manually, but you have 4 or 8 cores so it's not a big problem IMHO. The idea is that there is no difference between the instances you have among different servers, and the ones you have locally. Given that every instance can hold a set of unrelated keys, why centralizing it in some way if completely separated threads can lead to max performances, and simpler code? Note that 48 independent threads or 48 processes are actually the same thing more or less, but nothing prevents us from starting N threads at startup if needed for some reason in the future, each with completely separated state. I don't understand your parallel about starting N operating systems btw... Anyway, this is not a problem just with Redis. Programming languages like Ruby are a mess with the usual conception of threads. And most Ruby programmers also think far too high level to really deal with a low level stuff such as synchronization. A shared nothing model with message passing, and one interpreter per thread, could lead to the maximum benefit while retaining the more abstract interface to the programmer. Believe it or not the time for old-fashion multi thread programming is gone (I hope).

jacquesm 16y ago
Or even change the number of processes on demand.
shin_lao 16y ago
48 independent threads and 48 processes can be very different on some operating systems. A process is more heavyweight. You will - amongst other things - increase the charge in terms of handles. For example, 48 processes maintaining the same network connections is a waste of resources. I'm afraid you might hit a wall with your single threaded approach. This is definitely not the road we have chosen (but we wish the best of luck nonetheless! :) ).
- antirez 16y ago
  In a shared nothing environment you need 48 different sockets, distinct data structures, and so forth. Given the copy-on-write strategy of many OSes the overhead can really be minimal (you'll not get N copies of the executable in memory). The context switching should be more or less the same cost... what I mean is that it's trivial to turn the N proesses design into N independent threads design, as there is no problem with locking, but it is probably not a win.
- liuliu 16y ago
  For now redis only perfectly run on Linux; I think that process is not heavyweight at all. As my recall, thread on Linux is just process with shared resource.

by antirez 16y ago

jacquesm 16y ago

Or even change the number of processes on demand.

shin_lao 16y ago

48 independent threads and 48 processes can be very different on some operating systems. A process is more heavyweight. You will - amongst other things - increase the charge in terms of handles. For example, 48 processes maintaining the same network connections is a waste of resources. I'm afraid you might hit a wall with your single threaded approach. This is definitely not the road we have chosen (but we wish the best of luck nonetheless! :) ).

antirez 16y ago
In a shared nothing environment you need 48 different sockets, distinct data structures, and so forth. Given the copy-on-write strategy of many OSes the overhead can really be minimal (you'll not get N copies of the executable in memory). The context switching should be more or less the same cost... what I mean is that it's trivial to turn the N proesses design into N independent threads design, as there is no problem with locking, but it is probably not a win.
liuliu 16y ago
For now redis only perfectly run on Linux; I think that process is not heavyweight at all. As my recall, thread on Linux is just process with shared resource.