5 ms·

You can run multiple instances of redis on the same machine, which gets you parallelism from multiple processes instead of multiple threads. There's some extra

You can run multiple instances of redis on the same machine, which gets you parallelism from multiple processes instead of multiple threads. There's some extra deployment effort in doing so, but it's not generally a big issue. It also isn't exactly single threaded. It calls fork when it wants to persist data to disk, which is functionally similar to starting up a thread to do disk IO.

sciurus 7y ago
Going from a single processes holding all your data to having your data sharded across multiple processes can be more than a little effort. Also, it might now have any benefit for you. Imagine a certain key is particularly hot. Having one multithreaded redis process handling access to it might speed things up. Running multiple sharded redis processes won't, since only one of them will have that key.
- nitwit005 7y ago
  Most redis users are running multiple instances regardless. That some of them happen to be on the same machine is largely a detail. It would make a little difference if there was a single hot key, but that's a bit unusual. Typically there's some subset of keys that are hot, and you can get them to hash across instances. People also tend to cache those values on the clients, as banging on redis constantly is a waste.
  - derefr 7y ago
    > but that's a bit unusual It's pretty common when using a single Redis stream as a lightweight Kafka.
BubRoss 7y ago
Doesn't that end up duplicating lots of data and work since the processes are independent? They don't share data do they?
- tomjakubowski 7y ago
  fork() on Linux (and many other OSes) shares pages between the parent/child processes in a copy-on-write scheme
  - braindeath 7y ago
    That doesn’t mean multiple long running processes will share memory though unless they actually create a shared mapping, which I think the gp is referring to. In addition to tremendous memory waste this also kills cache performance.
    - tomjakubowski 7y ago
      I'd presume the persistence / I/O process would be pretty short lived: its job is to save a snapshot to disk so there's not much reason to let it live long.
      - BubRoss 7y ago
        How does that share data between multiple Redis processes?

sciurus 7y ago

Going from a single processes holding all your data to having your data sharded across multiple processes can be more than a little effort. Also, it might now have any benefit for you. Imagine a certain key is particularly hot. Having one multithreaded redis process handling access to it might speed things up. Running multiple sharded redis processes won't, since only one of them will have that key.

nitwit005 7y ago
Most redis users are running multiple instances regardless. That some of them happen to be on the same machine is largely a detail. It would make a little difference if there was a single hot key, but that's a bit unusual. Typically there's some subset of keys that are hot, and you can get them to hash across instances. People also tend to cache those values on the clients, as banging on redis constantly is a waste.
- derefr 7y ago
  > but that's a bit unusual It's pretty common when using a single Redis stream as a lightweight Kafka.

BubRoss 7y ago

Doesn't that end up duplicating lots of data and work since the processes are independent? They don't share data do they?

tomjakubowski 7y ago
fork() on Linux (and many other OSes) shares pages between the parent/child processes in a copy-on-write scheme
- braindeath 7y ago
  That doesn’t mean multiple long running processes will share memory though unless they actually create a shared mapping, which I think the gp is referring to. In addition to tremendous memory waste this also kills cache performance.
  - tomjakubowski 7y ago
    I'd presume the persistence / I/O process would be pretty short lived: its job is to save a snapshot to disk so there's not much reason to let it live long.
    - BubRoss 7y ago
      How does that share data between multiple Redis processes?