4 ms·

redis-trib.rb is finally ported to redis-cli. Thanks Salvatore! Stream data type looks very interesting as well. Is the goal to be competitive to Kafka-like us

by didip 8y ago

redis-trib.rb is finally ported to redis-cli. Thanks Salvatore! Stream data type looks very interesting as well. Is the goal to be competitive to Kafka-like use-case?

antirez 8y ago
Hello didip, I've a non-competitive style when the matter is software :-) I simply think that Redis streams could be very useful, in some way I borrowed Kafka ideas, in other ways I totally refused the Kafka implementation / approach, and in general Redis streams have a lot of cons and pro at the same time in a mixed way, due to the total difference between Kafka end Redis. So I think it's the users that should apply the right tools for the right jobs, I'll avoid any comparison. I just gave Kafka credits in the Redis doc because certain ideas are clearly from Kafka or popularized by Kafka.
- lossolo 8y ago
  Hello Salvatore, in which cases you would not use Redis streams and used Kafka/other solutions ? And in which cases you think Redis streams shine compared to other solutions ?
  - antirez 8y ago
    Hello lossolo, I hope that the community will figure this out. I'll try to make videos, blog posts and programming examples about Streams in Redis, but I don't like to make direct comparisons and I don't want to become a Kafka/other solution expert as well in order to make sure to make fair comparison :-) If I think at Redis Streams, I think they work very well in a set of domains. The king use case IMHO is time series. Having fast range queries is pretty interesting for this use case. Then the second one is events and event sourcing of all the kinds when the number of events-per-second is very sustained. However I've some feeling that also thinks like delayed jobs that are currently implemented on top of blocking lists may be rewritten to use streams and offer better overall features/operations.
    - sertorius 8y ago
      Just wanted to affirm this - I saw the streams announcement last year on the same day that I was having issues with Kafka and I implemented it into production for processing IoT time series data right away. It was probably a bit too early for production use, but it worked perfectly! One nice advantage of it vs. Kafka is that given that it's so lightweight, a Redis instance with streams can be dedicated to each customer with its own port and firewall setup and give you a really solid security story for clients who are paranoid about such things. (Kafka security is possible too of course...but harder to explain for a layman.) Thanks antirez - it's great. I've written a python Redis iterator that I'm looking forward to PR'ing to the python client to encapsulate the XREAD timestamp tracking bit and manage timeouts.
      - antirez 8y ago
        Thanks! Glad to hear a real world experience about it. And thanks for the Python client PR plan.

by didip 8y ago

redis-trib.rb is finally ported to redis-cli. Thanks Salvatore! Stream data type looks very interesting as well. Is the goal to be competitive to Kafka-like use-case?

antirez 8y ago

Hello didip, I've a non-competitive style when the matter is software :-) I simply think that Redis streams could be very useful, in some way I borrowed Kafka ideas, in other ways I totally refused the Kafka implementation / approach, and in general Redis streams have a lot of cons and pro at the same time in a mixed way, due to the total difference between Kafka end Redis. So I think it's the users that should apply the right tools for the right jobs, I'll avoid any comparison. I just gave Kafka credits in the Redis doc because certain ideas are clearly from Kafka or popularized by Kafka.

lossolo 8y ago
Hello Salvatore, in which cases you would not use Redis streams and used Kafka/other solutions ? And in which cases you think Redis streams shine compared to other solutions ?
- antirez 8y ago
  Hello lossolo, I hope that the community will figure this out. I'll try to make videos, blog posts and programming examples about Streams in Redis, but I don't like to make direct comparisons and I don't want to become a Kafka/other solution expert as well in order to make sure to make fair comparison :-) If I think at Redis Streams, I think they work very well in a set of domains. The king use case IMHO is time series. Having fast range queries is pretty interesting for this use case. Then the second one is events and event sourcing of all the kinds when the number of events-per-second is very sustained. However I've some feeling that also thinks like delayed jobs that are currently implemented on top of blocking lists may be rewritten to use streams and offer better overall features/operations.
  - sertorius 8y ago
    Just wanted to affirm this - I saw the streams announcement last year on the same day that I was having issues with Kafka and I implemented it into production for processing IoT time series data right away. It was probably a bit too early for production use, but it worked perfectly! One nice advantage of it vs. Kafka is that given that it's so lightweight, a Redis instance with streams can be dedicated to each customer with its own port and firewall setup and give you a really solid security story for clients who are paranoid about such things. (Kafka security is possible too of course...but harder to explain for a layman.) Thanks antirez - it's great. I've written a python Redis iterator that I'm looking forward to PR'ing to the python client to encapsulate the XREAD timestamp tracking bit and manage timeouts.
    - antirez 8y ago
      Thanks! Glad to hear a real world experience about it. And thanks for the Python client PR plan.