3 ms·

AWS reveals new Redis client

6 points by nihohit 3y ago

buzzdenver 3y ago
Can somebody eli5 how this works, or point me to some description? Let's say I'm pip installing it. How does the rust code turn into something that python can call?
- steveklabnik 3y ago
  It’s the same thing as when Python calls out to a module written in C. In this case they’re using https://crates.io/crates/pyo3 https://crates.io/crates/pyo3 to make the process easier.
bearjaws 3y ago
That's a lot of words for "we had to do this because of licensing".
- reconditerose 3y ago
  This has nothing to do with licensing. The motivation was that the various Redis clients are inconsistently being maintained, have different features, and perform differently. Having a core rust implementation handle the heavy lifting makes it easier to build the per-language bindings.
reconditerose 3y ago
For those that don't click through, the announcements is more of a preview, and has limited scope (only supports a subset of the commands and only support Python and Node). It's entirely open-source with a liberal license and you can find the source here: https://github.com/aws/glide-for-redis/ https://github.com/aws/glide-for-redis/. The main innovation that makes it interesting is that it uses a common rust core, built on top of redis-rs, to implement the heavy lifting of cluster topology and connection management. This code is much harder to implement correctly compared to the RESP protocol for simple commands. Higher level language wrappers interact with this core over a protobuf based protocol. The design is flexible, in the future it could support alternative backends (like memcached) and should be easily extensible for for new languages. I know the team is looking for folks interested in contributing.
kondro 3y ago
I wish they released a Rust-based client too. The current state of async-based Redis cluster compatible Rust clients that support pooling and decent connect/reconnect strategies is a bit of a mess.
- nihohit 3y ago
  it's based over redis-rs, and we're upstreaming all of the changes back. You can try using the fork - https://github.com/amazon-contributing/redis-rs https://github.com/amazon-contributing/redis-rs - but be warned that we don't publish a crate, you'll need to use it with the github hook, and that we might break APIs, so don't expect interface stability.
  - kondro 3y ago
    Good to know. Although that upstream commit ratio looks a bit scary. ;)

6 points by nihohit 3y ago

Can somebody eli5 how this works, or point me to some description? Let's say I'm pip installing it. How does the rust code turn into something that python can call?

steveklabnik 3y ago
It’s the same thing as when Python calls out to a module written in C. In this case they’re using https://crates.io/crates/pyo3 https://crates.io/crates/pyo3 to make the process easier.

bearjaws 3y ago

That's a lot of words for "we had to do this because of licensing".

reconditerose 3y ago
This has nothing to do with licensing. The motivation was that the various Redis clients are inconsistently being maintained, have different features, and perform differently. Having a core rust implementation handle the heavy lifting makes it easier to build the per-language bindings.

reconditerose 3y ago

For those that don't click through, the announcements is more of a preview, and has limited scope (only supports a subset of the commands and only support Python and Node). It's entirely open-source with a liberal license and you can find the source here: https://github.com/aws/glide-for-redis/ https://github.com/aws/glide-for-redis/. The main innovation that makes it interesting is that it uses a common rust core, built on top of redis-rs, to implement the heavy lifting of cluster topology and connection management. This code is much harder to implement correctly compared to the RESP protocol for simple commands. Higher level language wrappers interact with this core over a protobuf based protocol. The design is flexible, in the future it could support alternative backends (like memcached) and should be easily extensible for for new languages. I know the team is looking for folks interested in contributing.

kondro 3y ago

I wish they released a Rust-based client too. The current state of async-based Redis cluster compatible Rust clients that support pooling and decent connect/reconnect strategies is a bit of a mess.

nihohit 3y ago
it's based over redis-rs, and we're upstreaming all of the changes back. You can try using the fork - https://github.com/amazon-contributing/redis-rs https://github.com/amazon-contributing/redis-rs - but be warned that we don't publish a crate, you'll need to use it with the github hook, and that we might break APIs, so don't expect interface stability.
- kondro 3y ago
  Good to know. Although that upstream commit ratio looks a bit scary. ;)