4 ms·

Hey, I am one of the maintainers of Valkey, I'll try to answer. I think there is a few things I would like to see in the mid to short term. We're trying to mak

Hey, I am one of the maintainers of Valkey, I'll try to answer. I think there is a few things I would like to see in the mid to short term. We're trying to make a lot of the core datastructures more efficient (both in terms of memory and performance) as well as the main dictionary. Valkey 8.0 (or whatever first major version we have) will have lower overhead per key-value pair. Multithreading performance is nice, but as you mentioned most people don't need it. It gives folks a lot of runway if they need to scale but don't want to use clustering, and can also be a "quick fix" for certain types of P99 or higher latency spikes. Clustering is also really hard to use today, and a lot of the current folks want to fix that. It's a huge community pain point. Observability is another pain point I would like to improve. (Disclosure, I work at AWS as well) We see a lot of customers ask about "why did we see a performance issue", and Redis really doesn't provide a lot of introspection to diagnose those issues. Another big area, which maybe will work for redict, is we want better integration with other OSS projects, especially CNCF projects like envoy, open-telemetry, K8s. There are a lot of self-developed projects floating around, we're hoping we can pull all of this together to make a more cohesive project for people to use instead of what we see today. I think another big issue will be clients. I'm concerned Redis will try to inject a poison pill into the clients they own to make it so they can only talk to "official" Redis versions. Ultimately a small community will struggle to maintain a lot of clients, so I think we need a larger (and likely commercial) investment into keeping clients open. We will likely passively support redict with our clients, so they'll get that for free. We want to do all the cool feature stuff to (timeseries, JSON, bloom, RAG), but I would like to keep the core pretty clean.

fmajid 3y ago
Thank you for your work on Redis, Madelyn, and I'm sorry for the utterly unjustified abuse that's been thrown your way simply because you work at AWS. One feature that would be very helpful would be some form of analytics and a memory profiler. At one point I'd written a python library to iterate over a RDB to identify what the top keys consuming most memory were, but Redis has added a lot of complexity and new data types since, so I doubt it still works.
drewdevault 3y ago
We're excited to see valkey innovate in these areas! We wish you the best of luck and wish that we could have merged our forks, but alas. >Ultimately a small community will struggle to maintain a lot of clients, so I think we need a larger (and likely commercial) investment into keeping clients open I'm not sure that this is true, at least not the commercial part. Redis clients are pretty simple (compared to Redis itself, at least). Redict is leading an effort to encourage community forks of the official Redis clients, as well as sending patches downstream to third-party clients. We set as an explicit goal to fork the ecosystem as well; we've started this work with hiredict and don't intend to stop. I think there was some mention at some point about our two forks working together on maintenance of the protocol specification independently of Redis Ltd, which would be a good way to ensure that the clients remain broadly compatible with both of our forks.
- reconditerose 3y ago
  > I'm not sure that this is true, at least not the commercial part. Redis clients are pretty simple (compared to Redis itself, at least). I'll get on my soap box then. The redis client ecosystem is fractured and awful, and it's mostly Cluster's fault. Getting cluster right is really hard, and most clients do it inconsistently and get something wrong, if they tried at all (hiredict doesn't for example). > I think there was some mention at some point about our two forks working together on maintenance of the protocol specification independently of Redis Ltd, which would be a good way to ensure that the clients remain broadly compatible with both of our forks. Yeah, I'm still fully aligned with this.
  - drewdevault 3y ago
    >I'll get on my soap box then. The redis client ecosystem is fractured and awful, and it's mostly Cluster's fault. Getting cluster right is really hard, and most clients do it inconsistently and get something wrong, if they tried at all (hiredict doesn't for example). Fair point, I think it's prudent to recognize this as a weakness in the ecosystem. But I would also say that applies generally of cluster, sentinel, etc, in all respects. I know this is a focus area for valkey for that reason.
crabmusket 3y ago
Thanks for your work Madelyn and I wish you and the team the best of good fortune!

fmajid 3y ago

Thank you for your work on Redis, Madelyn, and I'm sorry for the utterly unjustified abuse that's been thrown your way simply because you work at AWS. One feature that would be very helpful would be some form of analytics and a memory profiler. At one point I'd written a python library to iterate over a RDB to identify what the top keys consuming most memory were, but Redis has added a lot of complexity and new data types since, so I doubt it still works.

drewdevault 3y ago

We're excited to see valkey innovate in these areas! We wish you the best of luck and wish that we could have merged our forks, but alas. >Ultimately a small community will struggle to maintain a lot of clients, so I think we need a larger (and likely commercial) investment into keeping clients open I'm not sure that this is true, at least not the commercial part. Redis clients are pretty simple (compared to Redis itself, at least). Redict is leading an effort to encourage community forks of the official Redis clients, as well as sending patches downstream to third-party clients. We set as an explicit goal to fork the ecosystem as well; we've started this work with hiredict and don't intend to stop. I think there was some mention at some point about our two forks working together on maintenance of the protocol specification independently of Redis Ltd, which would be a good way to ensure that the clients remain broadly compatible with both of our forks.

reconditerose 3y ago
> I'm not sure that this is true, at least not the commercial part. Redis clients are pretty simple (compared to Redis itself, at least). I'll get on my soap box then. The redis client ecosystem is fractured and awful, and it's mostly Cluster's fault. Getting cluster right is really hard, and most clients do it inconsistently and get something wrong, if they tried at all (hiredict doesn't for example). > I think there was some mention at some point about our two forks working together on maintenance of the protocol specification independently of Redis Ltd, which would be a good way to ensure that the clients remain broadly compatible with both of our forks. Yeah, I'm still fully aligned with this.
- drewdevault 3y ago
  >I'll get on my soap box then. The redis client ecosystem is fractured and awful, and it's mostly Cluster's fault. Getting cluster right is really hard, and most clients do it inconsistently and get something wrong, if they tried at all (hiredict doesn't for example). Fair point, I think it's prudent to recognize this as a weakness in the ecosystem. But I would also say that applies generally of cluster, sentinel, etc, in all respects. I know this is a focus area for valkey for that reason.

crabmusket 3y ago

Thanks for your work Madelyn and I wish you and the team the best of good fortune!