4 ms·

So this year on RedisConf Antirez demoed threaded version for Redis (only transport needs to be multi-threaded, core remains single threaded). Numbers were alre

So this year on RedisConf Antirez demoed threaded version for Redis (only transport needs to be multi-threaded, core remains single threaded). Numbers were already amazing. I will pick the community version of Redis any day over forks.

ksec 7y ago
So he changed his mind regarding threads? Will it actually be committed? or are they more or an experiment.
- xfalcox 7y ago
  It is live on unstable which should become stable by the end of year: https://github.com/antirez/redis/commits/unstable?after=c65347ab17d61a4118efdc4a3568bf71b088ab63+270 https://github.com/antirez/redis/commits/unstable?after=c653...
MuffinFlavored 7y ago
> only transport needs to be multi-threaded Could he have gone with a non-blocking approach like libuv?
- yazaddaruvala 7y ago
  Redis is single threaded because it heavily relies on non-blocking IO like libuv. The multithreaded option is also based on non-blocking IO. You can think of it as one Event Loop per CPU core, rather than only one Event Loop per computer.
  - gigatexal 7y ago
    Isn’t that basically what KeyDB does?
    - yazaddaruvala 7y ago
      I’m sure it’s the same. I was mostly outlining the pattern in general, not trying to imply it’s redis specific. A sibling post says that the keydb implementation also lets the multithreaded executor perform parsing. So I guess that is one difference.
drenvuk 7y ago
I feel like this take is somewhat unfair. Antirez was against threading until jdsully proved that it worked in exactly the same way that you're saying, multi-threaded transport with a single threaded core. In addition Keydb allows users to use ssds in addition to ram only when that is a paid feature for Redis Lab's enterprise support. jdsully spurred the implementation of two of the best changes that you might see in redis in the near future and you consider even using keydb pointless. Yea, ok.
- antirez 7y ago
  This is historically not correct, the threaded I/O branch was started 1.5 years ago, you can check the history online, it's all public. Moreover before me, and after me, a number of individuals did the same work many times, including Alibaba team, AWS team, and so forth. It's an obvious feature, the reason for not doing this, or doing this in a very limited fashion, is philosophical rather than technical. Btw Keydb way of doing threading has nothing to do with the trick used by Redis I/O threading AFAIK, of just fanning out to N threads only in the hot places.
jdsully 7y ago
The Redis version does not thread query parsing. Its limited to only IO which is why its performance is much lower.
- antirez 7y ago
  The Redis threaded I/O does parsing as well, but parsing does not change the obtained speedup a lot, so it is disabled by default (but you can enable it via config).

ksec 7y ago

So he changed his mind regarding threads? Will it actually be committed? or are they more or an experiment.

xfalcox 7y ago
It is live on unstable which should become stable by the end of year: https://github.com/antirez/redis/commits/unstable?after=c65347ab17d61a4118efdc4a3568bf71b088ab63+270 https://github.com/antirez/redis/commits/unstable?after=c653...

MuffinFlavored 7y ago

> only transport needs to be multi-threaded Could he have gone with a non-blocking approach like libuv?

yazaddaruvala 7y ago
Redis is single threaded because it heavily relies on non-blocking IO like libuv. The multithreaded option is also based on non-blocking IO. You can think of it as one Event Loop per CPU core, rather than only one Event Loop per computer.
- gigatexal 7y ago
  Isn’t that basically what KeyDB does?
  - yazaddaruvala 7y ago
    I’m sure it’s the same. I was mostly outlining the pattern in general, not trying to imply it’s redis specific. A sibling post says that the keydb implementation also lets the multithreaded executor perform parsing. So I guess that is one difference.

drenvuk 7y ago

I feel like this take is somewhat unfair. Antirez was against threading until jdsully proved that it worked in exactly the same way that you're saying, multi-threaded transport with a single threaded core. In addition Keydb allows users to use ssds in addition to ram only when that is a paid feature for Redis Lab's enterprise support. jdsully spurred the implementation of two of the best changes that you might see in redis in the near future and you consider even using keydb pointless. Yea, ok.

antirez 7y ago
This is historically not correct, the threaded I/O branch was started 1.5 years ago, you can check the history online, it's all public. Moreover before me, and after me, a number of individuals did the same work many times, including Alibaba team, AWS team, and so forth. It's an obvious feature, the reason for not doing this, or doing this in a very limited fashion, is philosophical rather than technical. Btw Keydb way of doing threading has nothing to do with the trick used by Redis I/O threading AFAIK, of just fanning out to N threads only in the hot places.

jdsully 7y ago

The Redis version does not thread query parsing. Its limited to only IO which is why its performance is much lower.

antirez 7y ago
The Redis threaded I/O does parsing as well, but parsing does not change the obtained speedup a lot, so it is disabled by default (but you can enable it via config).