8 ms·

I've always considered the single-threaded nature of Redis to be one of its greatest features.

moab 7y ago
Can you elaborate on why? As someone who works on parallel/concurrent algorithms and data structures a single-threaded system design seems anathema to most research over the past decade (maybe excluding H-Store, which uses local single-threadedness in an interesting way).
- nicolaslem 7y ago
  Not parent but the single threaded approach really shines when executing Lua scripts in Redis. Except for some very specific edge cases, the execution of a script is atomic and since no command is ever processed concurrently you can rule out data races. I am sure it would be possible to provide these guaranties while offering concurrent execution but most likely at the expense of a simple design.
  - jdsully 7y ago
    KeyDB provides that guarantee via a lock. However that does mean long running scripts don’t benefit from the multithreading. Ditto for modules. The goal is 100% Redis compatibility so I can’t compromise on atomicity.
  - avar 7y ago
    It shines right up until the moment someone writes a Lua command that takes more than a trivial amount of time, now your entire server is blocked in a pile-up of connections waiting on some relatively trivial amount of code being executed.
    - aflag 7y ago
      Couldn't you just rollback the change or fix the lua script?
      - thamer 7y ago
        Or just embrace modern hardware and use threads. This is the experience many people have running Redis in production with 21st-century hardware: https://twitter.com/rbranson/status/540565059337195520 https://twitter.com/rbranson/status/540565059337195520
        cal97g 7y ago
        That guy is saying that alternatives to redis are better, meanwhile he's still using redis..
        aflag 7y ago
        That's a different issue. If they're in that situation without any custom scripts and the latency is cropping up, they should consider using a different tool or changing their codebase.
- twic 7y ago
  This reminds me of LMAX, who found the fastest way to build their stock exchange was to make the core logic single-threaded, surrounded by multithreaded I/O: https://www.martinfowler.com/articles/lmax.html https://www.martinfowler.com/articles/lmax.html I believe other (grown-up/legacy!) exchanges work the same way. I wonder how much of the direction of concurrency research is driven by the fact that there is much more publishable work to be done in managing concurrency rather than avoiding it!
  - nvarsj 7y ago
    The disruptor isn’t single threaded at all. Maybe you mean it can pin a core to a consumer that just busy waits on the queue. That is a common technique used in very low latency systems when not running on a real time OS.
    - twic 7y ago
      The matching engine is single threaded. It uses disruptors to communicate with IO threads.
  - lllr_finger 7y ago
    Disruptor benefits from lock free access using a ring buffer. You can get amazing results with a single thread but that aspect isn't central to the pattern.
- nitwit005 7y ago
  You can run multiple instances of redis on the same machine, which gets you parallelism from multiple processes instead of multiple threads. There's some extra deployment effort in doing so, but it's not generally a big issue. It also isn't exactly single threaded. It calls fork when it wants to persist data to disk, which is functionally similar to starting up a thread to do disk IO.
  - sciurus 7y ago
    Going from a single processes holding all your data to having your data sharded across multiple processes can be more than a little effort. Also, it might now have any benefit for you. Imagine a certain key is particularly hot. Having one multithreaded redis process handling access to it might speed things up. Running multiple sharded redis processes won't, since only one of them will have that key.
    - nitwit005 7y ago
      Most redis users are running multiple instances regardless. That some of them happen to be on the same machine is largely a detail. It would make a little difference if there was a single hot key, but that's a bit unusual. Typically there's some subset of keys that are hot, and you can get them to hash across instances. People also tend to cache those values on the clients, as banging on redis constantly is a waste.
      - derefr 7y ago
        > but that's a bit unusual It's pretty common when using a single Redis stream as a lightweight Kafka.
  - BubRoss 7y ago
    Doesn't that end up duplicating lots of data and work since the processes are independent? They don't share data do they?
    - tomjakubowski 7y ago
femto113 7y ago
Only one thread can access the data at any given time, so it seems like most of the things you'd expect to be guaranteed by a single thread still are. I found this comment particularly interesting Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network. I wonder if anyone in the Redis ecosphere has explored a binary client server protocol, something that could be parsed/compiled on the client and then executed without parsing on the server, if the above is really true seems like that might offer even more perf gain than multithreading on the server.
- cheald 7y ago
  The human-readable/writable protocol is one of my favorite things about Redis, tbh. I can see cases where a really optimized system could benefit from a binary protocol, but I suspect it'd be a loss for most people.
  - axaxs 7y ago
    Why not just offer both?
    - femto113 7y ago
      That was my thinking as well, though taking a peek at the actual code suggests that there's a pretty deep expectation that the client is speaking strings, e.g. in code that handles the ZRANGE command[1] I see if (c->argc == 5 && !strcasecmp(c->argv[4]->ptr,"withscores")) and a quick grep suggests that's a common pattern % grep argv src/*.c | grep -c -e 'str\(case\)*cmp' 482 I guess this means someone would have to tackle creating an intermediate binary format first, rewriting the command handlers to expect that format, and then making client libraries that can produce the format. Perhaps still worth it in the end, but not trivial. [1] https://github.com/antirez/redis/blob/unstable/src/t_zset.c#L2422 https://github.com/antirez/redis/blob/unstable/src/t_zset.c#...
- smueller1234 7y ago
  From having played with/worked on profiling and optimizing Redis in the 2.6 timeframe, I can confirm that at least for small/simple operations, this is true, the data structure access is a small fraction of the cost. One related choice that Redis makes (or made at the time) is to rely extremely heavily on the malloc implementation, rather than doing work to manage it's memory internally. Even a very trivial, naive free list provided a modest speed-up, for example. There are a lot of these choices in the code base, largely owing to maintainability concerns (though antirez can surely speak for himself). Given how easy it is for an otherwise uninitiated C programmer such as myself to hack on it, I struggle to disagree with the prioritization. :)
atonse 7y ago
Is it because of the consistency guarantees you can get from single threaded operation? Because it seems like this guarantees that. It mostly parallelizes the command parsing and networking side. The actual core hash table is guarded by a global lock, so you could still get all those single-threaded guarantees.
- deleted 7y ago
  [deleted]
- JackFr 7y ago
  Won't it potentially allow out of order updates? Or am I missing something.
acroback 7y ago
Oh... Tell me more why being not multithreaded is a failure for Redis?

moab 7y ago

Can you elaborate on why? As someone who works on parallel/concurrent algorithms and data structures a single-threaded system design seems anathema to most research over the past decade (maybe excluding H-Store, which uses local single-threadedness in an interesting way).

nicolaslem 7y ago
Not parent but the single threaded approach really shines when executing Lua scripts in Redis. Except for some very specific edge cases, the execution of a script is atomic and since no command is ever processed concurrently you can rule out data races. I am sure it would be possible to provide these guaranties while offering concurrent execution but most likely at the expense of a simple design.
- jdsully 7y ago
  KeyDB provides that guarantee via a lock. However that does mean long running scripts don’t benefit from the multithreading. Ditto for modules. The goal is 100% Redis compatibility so I can’t compromise on atomicity.
- avar 7y ago
  It shines right up until the moment someone writes a Lua command that takes more than a trivial amount of time, now your entire server is blocked in a pile-up of connections waiting on some relatively trivial amount of code being executed.
  - aflag 7y ago
    Couldn't you just rollback the change or fix the lua script?
    - thamer 7y ago
      Or just embrace modern hardware and use threads. This is the experience many people have running Redis in production with 21st-century hardware: https://twitter.com/rbranson/status/540565059337195520 https://twitter.com/rbranson/status/540565059337195520
      - cal97g 7y ago
        That guy is saying that alternatives to redis are better, meanwhile he's still using redis..
      - aflag 7y ago
        That's a different issue. If they're in that situation without any custom scripts and the latency is cropping up, they should consider using a different tool or changing their codebase.
twic 7y ago
This reminds me of LMAX, who found the fastest way to build their stock exchange was to make the core logic single-threaded, surrounded by multithreaded I/O: https://www.martinfowler.com/articles/lmax.html https://www.martinfowler.com/articles/lmax.html I believe other (grown-up/legacy!) exchanges work the same way. I wonder how much of the direction of concurrency research is driven by the fact that there is much more publishable work to be done in managing concurrency rather than avoiding it!
- nvarsj 7y ago
  The disruptor isn’t single threaded at all. Maybe you mean it can pin a core to a consumer that just busy waits on the queue. That is a common technique used in very low latency systems when not running on a real time OS.
  - twic 7y ago
    The matching engine is single threaded. It uses disruptors to communicate with IO threads.
- lllr_finger 7y ago
  Disruptor benefits from lock free access using a ring buffer. You can get amazing results with a single thread but that aspect isn't central to the pattern.
nitwit005 7y ago
You can run multiple instances of redis on the same machine, which gets you parallelism from multiple processes instead of multiple threads. There's some extra deployment effort in doing so, but it's not generally a big issue. It also isn't exactly single threaded. It calls fork when it wants to persist data to disk, which is functionally similar to starting up a thread to do disk IO.
- sciurus 7y ago
  Going from a single processes holding all your data to having your data sharded across multiple processes can be more than a little effort. Also, it might now have any benefit for you. Imagine a certain key is particularly hot. Having one multithreaded redis process handling access to it might speed things up. Running multiple sharded redis processes won't, since only one of them will have that key.
  - nitwit005 7y ago
    Most redis users are running multiple instances regardless. That some of them happen to be on the same machine is largely a detail. It would make a little difference if there was a single hot key, but that's a bit unusual. Typically there's some subset of keys that are hot, and you can get them to hash across instances. People also tend to cache those values on the clients, as banging on redis constantly is a waste.
    - derefr 7y ago
      > but that's a bit unusual It's pretty common when using a single Redis stream as a lightweight Kafka.
- BubRoss 7y ago
  Doesn't that end up duplicating lots of data and work since the processes are independent? They don't share data do they?
  - tomjakubowski 7y ago

femto113 7y ago

Only one thread can access the data at any given time, so it seems like most of the things you'd expect to be guaranteed by a single thread still are. I found this comment particularly interesting Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network. I wonder if anyone in the Redis ecosphere has explored a binary client server protocol, something that could be parsed/compiled on the client and then executed without parsing on the server, if the above is really true seems like that might offer even more perf gain than multithreading on the server.

cheald 7y ago
The human-readable/writable protocol is one of my favorite things about Redis, tbh. I can see cases where a really optimized system could benefit from a binary protocol, but I suspect it'd be a loss for most people.
- axaxs 7y ago
  Why not just offer both?
  - femto113 7y ago
    That was my thinking as well, though taking a peek at the actual code suggests that there's a pretty deep expectation that the client is speaking strings, e.g. in code that handles the ZRANGE command[1] I see if (c->argc == 5 && !strcasecmp(c->argv[4]->ptr,"withscores")) and a quick grep suggests that's a common pattern % grep argv src/*.c | grep -c -e 'str\(case\)*cmp' 482 I guess this means someone would have to tackle creating an intermediate binary format first, rewriting the command handlers to expect that format, and then making client libraries that can produce the format. Perhaps still worth it in the end, but not trivial. [1] https://github.com/antirez/redis/blob/unstable/src/t_zset.c#L2422 https://github.com/antirez/redis/blob/unstable/src/t_zset.c#...
smueller1234 7y ago
From having played with/worked on profiling and optimizing Redis in the 2.6 timeframe, I can confirm that at least for small/simple operations, this is true, the data structure access is a small fraction of the cost. One related choice that Redis makes (or made at the time) is to rely extremely heavily on the malloc implementation, rather than doing work to manage it's memory internally. Even a very trivial, naive free list provided a modest speed-up, for example. There are a lot of these choices in the code base, largely owing to maintainability concerns (though antirez can surely speak for himself). Given how easy it is for an otherwise uninitiated C programmer such as myself to hack on it, I struggle to disagree with the prioritization. :)

atonse 7y ago

Is it because of the consistency guarantees you can get from single threaded operation? Because it seems like this guarantees that. It mostly parallelizes the command parsing and networking side. The actual core hash table is guarded by a global lock, so you could still get all those single-threaded guarantees.

deleted 7y ago
[deleted]
JackFr 7y ago
Won't it potentially allow out of order updates? Or am I missing something.

acroback 7y ago

Oh... Tell me more why being not multithreaded is a failure for Redis?