4 ms·

Gladly, Valkey has other benefits besides its more permissive license compared to Redis. One of them is multi-threading support, which makes it pretty easy for

Gladly, Valkey has other benefits besides its more permissive license compared to Redis. One of them is multi-threading support, which makes it pretty easy for someone like me—who has a bit of a skill issue with DevOps—to actually utilize all CPU cores for my caching layer.

antirez 1y ago
The I/O threading was initially implemented by myself, when I was still working on the core before rejoining, and is also available in Redis. So this is actually another case of the benefits ValKey got for being able to clone Redis, receiving money from the hyper-scalers that just want to maximize their revenues... They made certain improvements later, but Redis 8 (see the release notes in my blog post if you are curious) improved this stuff a lot, too. Also, Redis 8 contains a new data structure, Vector Sets, that allows to do many useful things, together with more probabilistic data structures, single hash fields expires, and many other stuff. It is not factually correct that ValKey has any features edge over Redis I believe. We will see in the next year which project takes the best development path: this is what really matters.
- reconditerose 1y ago
  The Valkey implementation of multi-threading is fundamentally different than what existed in Redis. The history dates back to work done in ElastiCache that was released as "Enhanced IO", https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-elasticache-for-redis-503-enhances-io-handling-to-boost-performance/ https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-el.... The version released in Redis could only do about 350k RPS because of poor memory locality of operations, the inability to do command processing while handling I/O, and the inability to offload much of the TCP read path. The new version in Valkey can achieve 1.2M RPS. "They made certain improvements later", should be "we threw away the old implementation and built a better one."
  - antirez 1y ago
    1. The fundamental idea was mine. Ideas can be improved, of course, and that's good. That was my point. However, allow me to say that Redis is fundamentally a "software of ideas" than anything else. Technologically it is far from impressive (Redis, ValKey, all the forks). 2. Redis 8 improves the same idea, too, released today. 3. If you claim [in a different comment here] you provided a lot of code to Redis, why you didn't send a pull request for that? So, you are practically saying you were using, at Amazon, all the BSD code we provided, but could not provide an important part of the code to us? You see how broken such model was? At least stop defending it. 4. We can now copy the implementation: the parts are reversed (the irony!), and your code is BSD as our was for 15 years. When we avoid doing things like that, is because we have issues with how certain things were made. 5. I don't understand the motivations of you and other AWS people commenting here today. You work for a company that is creating issues to the OSS ecosystem: this is hard to deny. You cloned (and, yes, the license allowed for it) the code of Redis, and work on it so that hyperscalers can continued to do what they used to do. We bring Redis back to AGPL, and you are here to do the interests of Amazon in the comments. Did you see me commenting your stuff, when you release your things, with comments like "ah! But this is unfair"? There is to make choices. I understand that it was cool to continue to work at a Redis fork, and part of the incredible thing open source is, is that forks survive in the hands of different teams (but design ideas can be misunderstood and projects may turn into other projects). So if you are happy to hack on ValKey, I hope you'll have the best experience out of it. But there is to make choices on how/when to interact.
    - _msw_ 1y ago
      This exchange makes me sad. I know we can do better. I don't understand why so many people think that it's impossible to have open source in your heart while working for a big company in your day job. I don't understand why people who have dedicated a lot of their time and emotional energy to keep open source ways alive and help build a community effort are attacked because they work for a company that needs to be made the villain in the narrative. Of course Redis is free to copy BSD licensed code that Valkey contributors add to the project [1]. I only wish that the blog post about this advancement in Redis would give some credit, rather than claiming "We also improved the performance of CRC64 calculations" [2]. We can all do better, and engage with one another with mutual respect and admiration for what has been freely given. [1] https://github.com/redis/redis/pull/13638 https://github.com/redis/redis/pull/13638 [2] https://redis.io/blog/redis-8-0-m03-is-out-even-more-performance-new-features/ https://redis.io/blog/redis-8-0-m03-is-out-even-more-perform...
- yjftsjthsd-h 1y ago
  > So this is actually another case of the benefits ValKey got for being able to clone Redis, receiving money from the hyper-scalers that just want to maximize their revenues... I don't think you want to sling that mud, unless Redis switched to being a nonprofit when I wasn't looking.
  - WD-42 1y ago
    They aren't really comparable and you know that.
    - yjftsjthsd-h 1y ago
      They are and you shouldn't claim bad faith without very good reason.
    - StreamingPanda 1y ago
      lmao they absolutely are

antirez 1y ago

The I/O threading was initially implemented by myself, when I was still working on the core before rejoining, and is also available in Redis. So this is actually another case of the benefits ValKey got for being able to clone Redis, receiving money from the hyper-scalers that just want to maximize their revenues... They made certain improvements later, but Redis 8 (see the release notes in my blog post if you are curious) improved this stuff a lot, too. Also, Redis 8 contains a new data structure, Vector Sets, that allows to do many useful things, together with more probabilistic data structures, single hash fields expires, and many other stuff. It is not factually correct that ValKey has any features edge over Redis I believe. We will see in the next year which project takes the best development path: this is what really matters.

reconditerose 1y ago
The Valkey implementation of multi-threading is fundamentally different than what existed in Redis. The history dates back to work done in ElastiCache that was released as "Enhanced IO", https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-elasticache-for-redis-503-enhances-io-handling-to-boost-performance/ https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-el.... The version released in Redis could only do about 350k RPS because of poor memory locality of operations, the inability to do command processing while handling I/O, and the inability to offload much of the TCP read path. The new version in Valkey can achieve 1.2M RPS. "They made certain improvements later", should be "we threw away the old implementation and built a better one."
- antirez 1y ago
  1. The fundamental idea was mine. Ideas can be improved, of course, and that's good. That was my point. However, allow me to say that Redis is fundamentally a "software of ideas" than anything else. Technologically it is far from impressive (Redis, ValKey, all the forks). 2. Redis 8 improves the same idea, too, released today. 3. If you claim [in a different comment here] you provided a lot of code to Redis, why you didn't send a pull request for that? So, you are practically saying you were using, at Amazon, all the BSD code we provided, but could not provide an important part of the code to us? You see how broken such model was? At least stop defending it. 4. We can now copy the implementation: the parts are reversed (the irony!), and your code is BSD as our was for 15 years. When we avoid doing things like that, is because we have issues with how certain things were made. 5. I don't understand the motivations of you and other AWS people commenting here today. You work for a company that is creating issues to the OSS ecosystem: this is hard to deny. You cloned (and, yes, the license allowed for it) the code of Redis, and work on it so that hyperscalers can continued to do what they used to do. We bring Redis back to AGPL, and you are here to do the interests of Amazon in the comments. Did you see me commenting your stuff, when you release your things, with comments like "ah! But this is unfair"? There is to make choices. I understand that it was cool to continue to work at a Redis fork, and part of the incredible thing open source is, is that forks survive in the hands of different teams (but design ideas can be misunderstood and projects may turn into other projects). So if you are happy to hack on ValKey, I hope you'll have the best experience out of it. But there is to make choices on how/when to interact.
  - _msw_ 1y ago
    This exchange makes me sad. I know we can do better. I don't understand why so many people think that it's impossible to have open source in your heart while working for a big company in your day job. I don't understand why people who have dedicated a lot of their time and emotional energy to keep open source ways alive and help build a community effort are attacked because they work for a company that needs to be made the villain in the narrative. Of course Redis is free to copy BSD licensed code that Valkey contributors add to the project [1]. I only wish that the blog post about this advancement in Redis would give some credit, rather than claiming "We also improved the performance of CRC64 calculations" [2]. We can all do better, and engage with one another with mutual respect and admiration for what has been freely given. [1] https://github.com/redis/redis/pull/13638 https://github.com/redis/redis/pull/13638 [2] https://redis.io/blog/redis-8-0-m03-is-out-even-more-performance-new-features/ https://redis.io/blog/redis-8-0-m03-is-out-even-more-perform...
yjftsjthsd-h 1y ago
> So this is actually another case of the benefits ValKey got for being able to clone Redis, receiving money from the hyper-scalers that just want to maximize their revenues... I don't think you want to sling that mud, unless Redis switched to being a nonprofit when I wasn't looking.
- WD-42 1y ago
  They aren't really comparable and you know that.
  - yjftsjthsd-h 1y ago
    They are and you shouldn't claim bad faith without very good reason.
  - StreamingPanda 1y ago
    lmao they absolutely are