9 ms·

Redis Stack

158 points by hadrianpaulo 5y ago

remote-dev 5y ago
Here is a video that outlines how to build a real-time stock watcher using Redis Stack: https://www.youtube.com/watch?v=LaiQFZ5bXaM https://www.youtube.com/watch?v=LaiQFZ5bXaM Worth a watch for sure
TipiKoivisto 5y ago
The first link on the page: Getting started gives 404, page not found. Hopefully the doesn't happen with the product itself.
- gkorland 5y ago
  Which link? on which page?
  - iRomain 5y ago
    https://imgur.com/a/JWwmb1x https://imgur.com/a/JWwmb1x
    - gkorland 5y ago
      thanks!
- Natfan 5y ago
  Someone's been silly with their links. They've linked the "getting started" text you linked to "https://redis.io/docs/stack/getting-started https://redis.io/docs/stack/getting-started", however in their navigation they have a separate "get started" link which points to "https://redis.io/docs/stack/get-started https://redis.io/docs/stack/get-started"
the_arun 5y ago
It would be great if ElastCache supports this
- gkorland 5y ago
  You can still get redis-stack on AWS see Redis Cloud https://redis.com/try-free/ https://redis.com/try-free/
- sciurus 5y ago
  This is made of separate codebases with new licenses, instead of being part of the MIT-licensed Redis, in order to prevent AWS from doing that.
- rsstack 5y ago
  Redis Stack is not Open Source, so that wouldn't be possible. AWS would have to reimplement all of the functionality, make their implementation Open Source, and keep adding new features to keep up.
ryannevius 5y ago
Here's their blog post about it: https://redis.com/blog/introducing-redis-stack/ https://redis.com/blog/introducing-redis-stack/
- gkorland 5y ago
  You might also want to watch this video https://www.youtube.com/watch?v=mUNFvyrsl8Q https://www.youtube.com/watch?v=mUNFvyrsl8Q
briandilley 5y ago
That time series though...
- gkorland 5y ago
  I would love to hear your feedback
a2800276 5y ago
What a crazy mess of services, so sad to see. redis was such a beautiful, elegant and pragmatic code base that was extremely effective at what it does. It looks like they are trying their hardest to make it more enterprisy to squeeze every last cent out.
- akmittal 5y ago
  Open source has changed a lot recently. Most people/companies do open source to make money. The initial phase is just to make the software popular enough so that a money-making business can be built around it. Frankly, I don't see anything bad with it. Those who don't want to spend money get it for free and those who can afford will pay for it.
  - fmajid 5y ago
    This is I-Can’t-Believe-It’s-Not-Open-Source™, not the real thing.
    - samhw 5y ago
      Is it? I mean, how? The core Redis software is still open source as it's always been, with the same (well, improved over time) functionality, whereas the closed-source for-profit components are not even 'paywalled features' but clearly separate higher-layer products building on the core (visualisation, ML, time series, full-text search, graph structures, &c). To me this is a paradigmatic example of well-done open-core.
- gkorland 5y ago
  It doesn't change Redis. redis-server will stay simple as it always been, redis-stack-server extends redis-server with addition functionality for those who need more low latency data access more than Redis K/V.
- eternityforest 5y ago
  Seems great to me. The AGPL style license is a bit much, but they have to make money somehow.
dikei 5y ago
You can really build pretty much anything on top of a K-V store, but I would hate to be vendor-locked into Redis ecosystem.
- rmbyrro 5y ago
  Just develop for an interface, instead of an implementation. Migraring data is no fun, but much more straightforward than rewriting a good part of your code when you decide to switch database.
  - rco8786 5y ago
    Is there any example of this ever working in practice in the history of computing? As in, we built a large, stable service with datastore X, but we were forward looking enough to make sure that we only "developed to an interface" and we successfully swapped to datastore Y without significant code changes?
    - rmbyrro 5y ago
      The project I'm currently working uses AWS DynamoDB as primary DB. Since it's proprietary, we ensure the AWS-specific APIs are contained in small and specific part of the codebase. For instance, to create a "User" we need the PutItem API from DynamoDB [1]. Similarly, to retrieve a "User", there's the GetItem API [2]. Instead of making references to these APIs all over our codebase, we have a single `db-interface` module, which implements `get_user` and `create_user` functions. Each of these functions has an interface: they expect specific arguments with corresponding types. This interface is modeled against our data domain, not DynamoDB data domain quirks. Inside the `db-interface` module, we implement the conversion from our data domain to Dynamo's. We also have general-purpose functions, like `create_item` and `get_item`, so that `get_user` and `create_user` are pretty much wrappers and only serve the purpose of defining the interface for interacting with the "User" data object. The rest of our code only interacts with our internal interface, never with DynamoDB APIs directly. If we were to switch database, we only need to re-implement the general-purpose functions (e.g. `create_item`, `get_item`). May sound like a lot of work, but it took only a handful of days to implement the entire interface mapping everything we needed from DynamoDB. Hope this helps clarifying a bit the application of this concept (develop for an interface, not an implementation) in the context of interacting with databases. [1] https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen... [2] https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...
pbowyer 5y ago
Redis Stack sounds like it provides the same features that ArangoDB does. Anyone done a comparison?
pachico 5y ago
True, you can always run vanilla Redis. Having said this, it really gives me the sensation that Redis is trying to solve too many use cases, turning into some sort of Swiss knife. The reality of Swiss knives is that they don't excel in anything of the things they do.
- throwaway6532 5y ago
  >The reality of Swiss knives is that they don't excel in anything of the things they do. Except the one thing that do excel at: being swiss army knives.
- gkorland 5y ago
  We feel like the most important part required from a low latency DB, Redis "Vanilla" and Redis Enterprise already cover. Now, we can leverage this amazing platform to provide a real low latency multi model DB on top of it.
  - packetlost 5y ago
    you can really do a lot in terms of building structured query engines on top of a simple performant key:value store
- aelzeiny 5y ago
  It's always been a Swiss Army Knife. It's a single-threaded-in-memory caching, pubSubing, streaming, distributed-locking beast.
qwertox 5y ago
Around two months ago I was excited to learn about RedisJSON, only to get to know that there is no ARM-based Docker image available for installing it on a Raspberry Pi.
- psychoslave 5y ago
  What is preventing you to contribute and create it?
  - burnished 5y ago
    Probably some combination of knowledge, experience, time and interest. Frankly it sounds like kind of an intimidating ask. Do you have special knowledge that makes this seem simple, or is it actually not challenging once you really get all the pieces involved? I'd love to be confident that I could see a statement like the one you replied to and immediately execute on it, so if you have any tips to share please do!
    - psychoslave 5y ago
      Well, to start, have you tried to send information in their issue tracker? Maybe my answer sounded more harsh in tone than what I meant, sorry for that. I red the message as "Hey they didn’t cover my specific case!", and I felt like this was unfair to ask an opensource project to cover each corner case upfront and not even consider a lacking feature as something that can be reported or maybe even contributed back.
      - burnished 5y ago
        Nope, I wasn't the top level comment. Yes, it came off sharp, it would have discouraged me from contributing. Sometimes people don't really get that the process is open to all and need to be invited, I think it can be good to try and see stuff like this as an opportunity to extend that invitation. I think you misread this one because it is basically "excitement about new thing -> implied disappointment at not getting to use new thing", and I can't find an ask in that.
pluc 5y ago
What is the logic behind separating those from the core server? Binary size? Automatic configuration? "Wow this has a lot of features" vs "wow what are all these extra things I need to sift through to figure out what I need", no?
- gkorland 5y ago
  All the above :)
mrwnmonm 5y ago
I don't understand why there is timeseries solution there, usually people who use timeseries work with many Gigas of storage, are we gonna pay for RAM to do that?
- gkorland 5y ago
  First it all depends on your latency requirements and TP, if you need your TimeSeries DB to support high ingest TP and low query latency a RAM based is almost the only way you can go. Second, many Gigas is what you get from any small size server... And, on top of that Redis/TimeSeries supports an almost linear scale out which allows your memory to increase dynamically according to your needs. Last, Redis Enterprise (& Cloud) add Redis on Flash support extending your RAM to your local Flash storage.
- rsstack 5y ago
  I have run Redis clusters, both with and without Redis on Flash, with hundreds of GBs of data. If you need the performance, it's doable and not prohibitively expensive.
denormalized 5y ago
I want to do a sentiment analysis on Redis-related HN comments pre and post antirez departure.
- gkorland 5y ago
  That can be a very interesting project :) I would even consider using RedisAI to run the model...
Lio 5y ago
It’s a shame they don’t offer first class Ruby support. I could see myself using some this stuff. Is this something Redis are likely to offer in future does anyone know?
- rsstack 5y ago
  > they don’t offer first class Ruby support It's a datastore. It has clients for many languages, including a recommended one for Ruby. What support are you missing here?
  - scoopertrooper 5y ago
    To use the extra features that come with Redis Stack, you need a client that supports those features. According to the site posted, the Ruby client doesn't support these features. So it appears that you are the one that is missing something.
    - samhw 5y ago
      > So it appears that you are the one that is missing something. They were asking what library support was missing for that person's use case (which your question answers, but I think the snark is based on a misunderstanding).
- gkorland 5y ago
  You're right, we would love to provide a much better experience in Ruby it was just a matter of priorities... BTW, all the modules already have ruby based clients developed by the community. e.g. * https://github.com/npezza93/redi_search https://github.com/npezza93/redi_search * https://github.com/vachhanihpavan/rejson-rb https://github.com/vachhanihpavan/rejson-rb * https://github.com/dzunk/redis-time-series https://github.com/dzunk/redis-time-series You can see on each modules docs the list of clients developed by the community. e.g. https://redis.io/docs/stack/graph/clients/ https://redis.io/docs/stack/graph/clients/
  - Lio 5y ago
faizshah 5y ago
The redisgraph functionality is really cool, it makes it an interesting choice over SQLite for small data science projects. The search functionality also is helpful for data science projects. Does anyone know what this RSAL license’s impact is for internal tools (e.g log analysis)?
- rsstack 5y ago
  RSAL 1.1(b): "[Licensor hereby grants to You a ... license ... to:] use the Software, or your Modifications, only as part of Your Application, but not in connection with any Database Product that is distributed or otherwise made available by any third party" As long as this log analysis is part of the production environment of your application, and you don't resell it as a tool, that should be fine.
aynyc 5y ago
This is pretty much the direction of all specialty database providers (MongoDB, Redis) are moving into. Everyone wants to be able to query datasets across enterprise. I bet is Redis Stack will soon querying against support Parquet or ORC files. Then connector to traditional databases like PG, MySQL, Oracle, etc.. Throw in some AI modeling. In a way, this is a marketing play to stay relevant in the enterprise space. Redis as a pure in-memory store (caching) engine just doesn't make as much money as "analytic system". I don't have a problem with this as they have to make money somehow.
ThinkBeat 5y ago
I wish product would be small sharp tools that does something well and sticks to it, iterating on making it faster,more stable, better documented, easier to setup and configure. Instead a lot of product start out that way and then if it becomes popular scope creep rears it ugly head and it is extended to be mediocre at many things. I find far too many developers once they start investing time in a product they want it to do all sorts of other things that there are allready tools that do it well.
manishsharan 5y ago
I have used Redis TimeSeries a few years ago and I found it easy to use and very performant for my needs. I think it is still chugging along at my last employer. The development team is already familiar with Redis and therefore found TimeSeries simple and easy to use. None of us wanted to use ( and learn) a dedicated TimeSeries database API and business did not want an expanded footprint to maintain. So Redis TimeSeries was an excellent solution for us
renonce 5y ago
The only thing I hate about Redis is that all data has to stay in memory... Well I know that's obvious, but Redis is nearly the perfect solution in terms of performance. Unlike disk-based databases, Redis has consistently optimal performance on the order of millions of queries per second. And horizontal scaling is trivial with Redis Cluster (it involves running multiple Redis instances and many Redis service providers don't support it - I hope to have an easier way of running cluster though). Given that memory is cheap it's quite plausible to store an entire database in a single Redis instance. I do hope that Redis finds its way to replace conventional databases. I think we just need an asynchronous `FLUSH` command, which blocks until the next `fsync`, and that should get us closer to ACID compliance.

158 points by hadrianpaulo 5y ago

Here is a video that outlines how to build a real-time stock watcher using Redis Stack: https://www.youtube.com/watch?v=LaiQFZ5bXaM https://www.youtube.com/watch?v=LaiQFZ5bXaM Worth a watch for sure

TipiKoivisto 5y ago

The first link on the page: Getting started gives 404, page not found. Hopefully the doesn't happen with the product itself.

gkorland 5y ago
Which link? on which page?
- iRomain 5y ago
  https://imgur.com/a/JWwmb1x https://imgur.com/a/JWwmb1x
  - gkorland 5y ago
    thanks!
Natfan 5y ago
Someone's been silly with their links. They've linked the "getting started" text you linked to "https://redis.io/docs/stack/getting-started https://redis.io/docs/stack/getting-started", however in their navigation they have a separate "get started" link which points to "https://redis.io/docs/stack/get-started https://redis.io/docs/stack/get-started"

the_arun 5y ago

It would be great if ElastCache supports this

gkorland 5y ago
You can still get redis-stack on AWS see Redis Cloud https://redis.com/try-free/ https://redis.com/try-free/
sciurus 5y ago
This is made of separate codebases with new licenses, instead of being part of the MIT-licensed Redis, in order to prevent AWS from doing that.
rsstack 5y ago
Redis Stack is not Open Source, so that wouldn't be possible. AWS would have to reimplement all of the functionality, make their implementation Open Source, and keep adding new features to keep up.

ryannevius 5y ago

Here's their blog post about it: https://redis.com/blog/introducing-redis-stack/ https://redis.com/blog/introducing-redis-stack/

gkorland 5y ago
You might also want to watch this video https://www.youtube.com/watch?v=mUNFvyrsl8Q https://www.youtube.com/watch?v=mUNFvyrsl8Q

briandilley 5y ago

That time series though...

gkorland 5y ago
I would love to hear your feedback

a2800276 5y ago

What a crazy mess of services, so sad to see. redis was such a beautiful, elegant and pragmatic code base that was extremely effective at what it does. It looks like they are trying their hardest to make it more enterprisy to squeeze every last cent out.

akmittal 5y ago
Open source has changed a lot recently. Most people/companies do open source to make money. The initial phase is just to make the software popular enough so that a money-making business can be built around it. Frankly, I don't see anything bad with it. Those who don't want to spend money get it for free and those who can afford will pay for it.
- fmajid 5y ago
  This is I-Can’t-Believe-It’s-Not-Open-Source™, not the real thing.
  - samhw 5y ago
    Is it? I mean, how? The core Redis software is still open source as it's always been, with the same (well, improved over time) functionality, whereas the closed-source for-profit components are not even 'paywalled features' but clearly separate higher-layer products building on the core (visualisation, ML, time series, full-text search, graph structures, &c). To me this is a paradigmatic example of well-done open-core.
gkorland 5y ago
It doesn't change Redis. redis-server will stay simple as it always been, redis-stack-server extends redis-server with addition functionality for those who need more low latency data access more than Redis K/V.
eternityforest 5y ago
Seems great to me. The AGPL style license is a bit much, but they have to make money somehow.

dikei 5y ago

You can really build pretty much anything on top of a K-V store, but I would hate to be vendor-locked into Redis ecosystem.

rmbyrro 5y ago
Just develop for an interface, instead of an implementation. Migraring data is no fun, but much more straightforward than rewriting a good part of your code when you decide to switch database.
- rco8786 5y ago
  Is there any example of this ever working in practice in the history of computing? As in, we built a large, stable service with datastore X, but we were forward looking enough to make sure that we only "developed to an interface" and we successfully swapped to datastore Y without significant code changes?
  - rmbyrro 5y ago
    The project I'm currently working uses AWS DynamoDB as primary DB. Since it's proprietary, we ensure the AWS-specific APIs are contained in small and specific part of the codebase. For instance, to create a "User" we need the PutItem API from DynamoDB [1]. Similarly, to retrieve a "User", there's the GetItem API [2]. Instead of making references to these APIs all over our codebase, we have a single `db-interface` module, which implements `get_user` and `create_user` functions. Each of these functions has an interface: they expect specific arguments with corresponding types. This interface is modeled against our data domain, not DynamoDB data domain quirks. Inside the `db-interface` module, we implement the conversion from our data domain to Dynamo's. We also have general-purpose functions, like `create_item` and `get_item`, so that `get_user` and `create_user` are pretty much wrappers and only serve the purpose of defining the interface for interacting with the "User" data object. The rest of our code only interacts with our internal interface, never with DynamoDB APIs directly. If we were to switch database, we only need to re-implement the general-purpose functions (e.g. `create_item`, `get_item`). May sound like a lot of work, but it took only a handful of days to implement the entire interface mapping everything we needed from DynamoDB. Hope this helps clarifying a bit the application of this concept (develop for an interface, not an implementation) in the context of interacting with databases. [1] https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen... [2] https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html https://docs.aws.amazon.com/amazondynamodb/latest/APIReferen...

pbowyer 5y ago

Redis Stack sounds like it provides the same features that ArangoDB does. Anyone done a comparison?

pachico 5y ago

True, you can always run vanilla Redis. Having said this, it really gives me the sensation that Redis is trying to solve too many use cases, turning into some sort of Swiss knife. The reality of Swiss knives is that they don't excel in anything of the things they do.

throwaway6532 5y ago
>The reality of Swiss knives is that they don't excel in anything of the things they do. Except the one thing that do excel at: being swiss army knives.
gkorland 5y ago
We feel like the most important part required from a low latency DB, Redis "Vanilla" and Redis Enterprise already cover. Now, we can leverage this amazing platform to provide a real low latency multi model DB on top of it.
- packetlost 5y ago
  you can really do a lot in terms of building structured query engines on top of a simple performant key:value store
aelzeiny 5y ago
It's always been a Swiss Army Knife. It's a single-threaded-in-memory caching, pubSubing, streaming, distributed-locking beast.

qwertox 5y ago

Around two months ago I was excited to learn about RedisJSON, only to get to know that there is no ARM-based Docker image available for installing it on a Raspberry Pi.

psychoslave 5y ago
What is preventing you to contribute and create it?
- burnished 5y ago
  Probably some combination of knowledge, experience, time and interest. Frankly it sounds like kind of an intimidating ask. Do you have special knowledge that makes this seem simple, or is it actually not challenging once you really get all the pieces involved? I'd love to be confident that I could see a statement like the one you replied to and immediately execute on it, so if you have any tips to share please do!
  - psychoslave 5y ago
    Well, to start, have you tried to send information in their issue tracker? Maybe my answer sounded more harsh in tone than what I meant, sorry for that. I red the message as "Hey they didn’t cover my specific case!", and I felt like this was unfair to ask an opensource project to cover each corner case upfront and not even consider a lacking feature as something that can be reported or maybe even contributed back.
    - burnished 5y ago
      Nope, I wasn't the top level comment. Yes, it came off sharp, it would have discouraged me from contributing. Sometimes people don't really get that the process is open to all and need to be invited, I think it can be good to try and see stuff like this as an opportunity to extend that invitation. I think you misread this one because it is basically "excitement about new thing -> implied disappointment at not getting to use new thing", and I can't find an ask in that.

pluc 5y ago

What is the logic behind separating those from the core server? Binary size? Automatic configuration? "Wow this has a lot of features" vs "wow what are all these extra things I need to sift through to figure out what I need", no?

gkorland 5y ago
All the above :)

mrwnmonm 5y ago

I don't understand why there is timeseries solution there, usually people who use timeseries work with many Gigas of storage, are we gonna pay for RAM to do that?

gkorland 5y ago
First it all depends on your latency requirements and TP, if you need your TimeSeries DB to support high ingest TP and low query latency a RAM based is almost the only way you can go. Second, many Gigas is what you get from any small size server... And, on top of that Redis/TimeSeries supports an almost linear scale out which allows your memory to increase dynamically according to your needs. Last, Redis Enterprise (& Cloud) add Redis on Flash support extending your RAM to your local Flash storage.
rsstack 5y ago
I have run Redis clusters, both with and without Redis on Flash, with hundreds of GBs of data. If you need the performance, it's doable and not prohibitively expensive.

denormalized 5y ago

I want to do a sentiment analysis on Redis-related HN comments pre and post antirez departure.

gkorland 5y ago
That can be a very interesting project :) I would even consider using RedisAI to run the model...

Lio 5y ago

It’s a shame they don’t offer first class Ruby support. I could see myself using some this stuff. Is this something Redis are likely to offer in future does anyone know?

rsstack 5y ago
> they don’t offer first class Ruby support It's a datastore. It has clients for many languages, including a recommended one for Ruby. What support are you missing here?
- scoopertrooper 5y ago
  To use the extra features that come with Redis Stack, you need a client that supports those features. According to the site posted, the Ruby client doesn't support these features. So it appears that you are the one that is missing something.
  - samhw 5y ago
    > So it appears that you are the one that is missing something. They were asking what library support was missing for that person's use case (which your question answers, but I think the snark is based on a misunderstanding).
gkorland 5y ago
You're right, we would love to provide a much better experience in Ruby it was just a matter of priorities... BTW, all the modules already have ruby based clients developed by the community. e.g. * https://github.com/npezza93/redi_search https://github.com/npezza93/redi_search * https://github.com/vachhanihpavan/rejson-rb https://github.com/vachhanihpavan/rejson-rb * https://github.com/dzunk/redis-time-series https://github.com/dzunk/redis-time-series You can see on each modules docs the list of clients developed by the community. e.g. https://redis.io/docs/stack/graph/clients/ https://redis.io/docs/stack/graph/clients/
- Lio 5y ago

faizshah 5y ago

The redisgraph functionality is really cool, it makes it an interesting choice over SQLite for small data science projects. The search functionality also is helpful for data science projects. Does anyone know what this RSAL license’s impact is for internal tools (e.g log analysis)?

rsstack 5y ago
RSAL 1.1(b): "[Licensor hereby grants to You a ... license ... to:] use the Software, or your Modifications, only as part of Your Application, but not in connection with any Database Product that is distributed or otherwise made available by any third party" As long as this log analysis is part of the production environment of your application, and you don't resell it as a tool, that should be fine.

aynyc 5y ago

This is pretty much the direction of all specialty database providers (MongoDB, Redis) are moving into. Everyone wants to be able to query datasets across enterprise. I bet is Redis Stack will soon querying against support Parquet or ORC files. Then connector to traditional databases like PG, MySQL, Oracle, etc.. Throw in some AI modeling. In a way, this is a marketing play to stay relevant in the enterprise space. Redis as a pure in-memory store (caching) engine just doesn't make as much money as "analytic system". I don't have a problem with this as they have to make money somehow.

ThinkBeat 5y ago

I wish product would be small sharp tools that does something well and sticks to it, iterating on making it faster,more stable, better documented, easier to setup and configure. Instead a lot of product start out that way and then if it becomes popular scope creep rears it ugly head and it is extended to be mediocre at many things. I find far too many developers once they start investing time in a product they want it to do all sorts of other things that there are allready tools that do it well.

manishsharan 5y ago

I have used Redis TimeSeries a few years ago and I found it easy to use and very performant for my needs. I think it is still chugging along at my last employer. The development team is already familiar with Redis and therefore found TimeSeries simple and easy to use. None of us wanted to use ( and learn) a dedicated TimeSeries database API and business did not want an expanded footprint to maintain. So Redis TimeSeries was an excellent solution for us

renonce 5y ago

The only thing I hate about Redis is that all data has to stay in memory... Well I know that's obvious, but Redis is nearly the perfect solution in terms of performance. Unlike disk-based databases, Redis has consistently optimal performance on the order of millions of queries per second. And horizontal scaling is trivial with Redis Cluster (it involves running multiple Redis instances and many Redis service providers don't support it - I hope to have an easier way of running cluster though). Given that memory is cheap it's quite plausible to store an entire database in a single Redis instance. I do hope that Redis finds its way to replace conventional databases. I think we just need an asynchronous `FLUSH` command, which blocks until the next `fsync`, and that should get us closer to ACID compliance.