12 ms·

Wow, the reasons why Redis commands API suck in Andy's video (linked in the post) are the weakest ever. It is possible to make a case against the Redis API (I w

Wow, the reasons why Redis commands API suck in Andy's video (linked in the post) are the weakest ever. It is possible to make a case against the Redis API (I would not agree of course but... it's totally legitimate), but you gotta have stronger arguments than those, particularly if you are a teacher of some kind. Especially: you need to be somewhat fluent in Redis and how developers use Redis in order to understand why so many people like it, and then elaborate what it's wrong about it (if you believe there is something wrong). The video shows a general feeling of "I don't really use / know this, but I don't like how NON-SQL it is".

nojito 2y ago
SQL is king and history has shown non-sql languages are not good which causes many non-sql DBMS's to adopt sql eventually.
- antirez 2y ago
  Many non-SQL DBs had query languages that were broken Javascript-ish versions of SQL. Of course, this is wrong, and people will eventually adopt SQL instead. But if your data model isn't anything like relational DBs, non-SQL makes a ton of sense. OP seems to miss exactly this, that the Redis query language is shaped on the Redis data model, that is basically alien to the relational model. The idea behind Redis data model is that "describe data" then "query those data in random ways" is conceptually nice but practically will not model very well many use cases. SQL databases plagued tech with performance issues for decades because of that. So Redis says instead: you need to put your data thinking about fundamental things like data structures and access times and the way you'll need those data back. And the API reflects this. You don't have to automatically agree with that. But you have to understand that, then provide your "I'm against" arguments. Especially if you are in front of young people listening to you.
  - im_down_w_otp 2y ago
    Agreed. Many noSQL-boom-era databases eventually bolted on a SQL-esque layer, but that was also because they were mostly also all targeting "enterprise database" use cases and customers who both expected that and whose use cases largely fit with it. So, there was a lot of pressure to conform to norms when the advantage of not doing so wasn't immediately self-evident. We have a database [1] and query language [2] that's tailored to storing & querying trace/telemetry data produced by different layers and components of cyber-physical systems for systems engineers to analyze, verify, and validate what a complex system is doing. It's not quite a traditional relational problem. It's not quite a traditional time series problem. It's not quite a traditional graph problem. Addressing the way that systems engineers think about their domain in an effective way required coming up with something different. Are there caveats and rough edges? Sure. But, they're a lot less pernicious and onerous than the alternative of trying to leverage a bunch of ill-fitting menageries of different solutions. Redis is fit-for-purpose. So, it makes sense that its query interface would also express that. [1] https://docs.auxon.io/modality/ https://docs.auxon.io/modality/ [2] https://docs.auxon.io/speqtr/ https://docs.auxon.io/speqtr/
  - nojito 2y ago
    >But if your data model isn't anything like relational DBs, non-SQL makes a ton of sense. OP seems to miss exactly this, that the Redis query language is shaped on the Redis data model, that is basically alien to the relational model. Sure...but all roads lead back to SQL eventually. Another recent example also mentioned in the OP is BigTable adopting SQL.
    - threeseed 2y ago
      > but all roads lead back to SQL eventually No it doesn't. SQL is designed for relational databases. For other forms i.e. JSON, Graph, Key/Value they all use other query languages.
    - physicles 2y ago
      If you write code that uses a hash map, would you insist on using SQL to query it? This makes no sense.
- anovick 2y ago
  SQL (and RDBMS in general) has its limitations, particularly with regards to recursive operations. An extended Datalog[1] can provide performance optimizations not available to RDBMS. [1]: https://dl.acm.org/doi/10.1145/3639271 https://dl.acm.org/doi/10.1145/3639271
- KronisLV 2y ago
  For what it's worth, SQL kind of sucks. It's just the de facto choice because it's extremely widespread and good enough for 80% of the use cases out there and what's missing can be kludged on top of it, either by specific DB vendors, or by various extensions. It's not too hard to come up with alternatives that improve upon individual aspects of SQL like https://prql-lang.org/ https://prql-lang.org/ but the barrier of entry is about as high as trying to make a huge social media network, most attempts will remain niche. Then again, most software kind of sucks, it's just that some of it also works. For example, the Linux FHS reads like an overcomplicated structure that is the way it is for historical reasons, but works in practice.
fforflo 2y ago
I've been working on something Redis-y over the holidays, and it has reinforced my view that it's the epitome of a 20%- 80% tool. I've always used the 20%, but anything beyond that sounds useless unless you've encountered the requirement in a production environment. The challenges Redis has been solving for years, never really touched the research/academic community (even the 20%). Even in the various taxonomies of DBS in the research literature, Redis was mentioned with a wave of the hand as an "in-memory" database, which undersells the important (for me) part of the "data structure" server. Putting the "database" after Redis could be a marketing misstep. Because it puts you in the is-it-sql territory. TL;DR: Redis is mostly appreciated by practitioners (web) developers. Academics find it lacking a theoretical foundation, so... meh.
- Tanjreeve 2y ago
  Developers know it's limits. Or you have developers with vague "scaling issues" or "buggy caching" who don't understand why they have them or suddenly start suffering from them at inconvenient moments.
nicoritschel 2y ago
With all due respect, the linked video was pretty fair. It didn't imply not to use Redis, just not as a primary datastore. I don't think folks work with Redis out of fondness for the model, but because it's the least worst datastore for caching, lightweight message broker, and simple realtime things like counters.
- antirez 2y ago
  Talking about the broken API argument here. Also Redis is particularly useful exactly in other situations compared to what OP says. Leaderboards style use cases with sorted sets are killer applications (super hard to model with SQL) of the data structure server thing. Apparently OP does not understand this and says "simple GET/SET" is what you should use Redis for. Redis has probabilistic data structures, the ability to implement complex queueing patterns, and so forth. That's where the value is. Otherwise we would still be just with Memcached without caring about Redis. Another killer app was Twitter initial use case (then they used it for pretty much everything): to cache latest N Tweets, using capped lists. I could continue forever. So OP argoment is flawed IMHO, for the above arguments, not fair. When you talk to students you need to make your homeworks. Really understand the system you are talking and provide a realistic image of it. Then, yes, if you want, criticize it as much as you want, with grounded arguments. You know what? I re-read this comment and it's embarassing I ever have to write this, because after 15 years of Redis history at such scale and popularity, pretty much everybody that was seriously exposed to Redis knows those stuff. Is tech culture really degraded so much that we have to restate the obvious? Do I really need to explain GET/SET is not exactly where Redis shines after 15 years of half the Internet used all the kind of Redis patterns?
  - memhole 2y ago
    What are your thoughts about Rails switching to SQLite from Redis? I've only used Redis to store session data and cache app data. So my opinions are pretty limited and mostly positive. https://rubyonrails.org/2024/11/7/rails-8-no-paas-required https://rubyonrails.org/2024/11/7/rails-8-no-paas-required
    - 2y ago
tayo42 2y ago
> Andy's video (linked in the post) Is there a "to long didnt watch" summary any one knows of? I hate videos, but am curious lol
- jghn 2y ago
  Right there with you. The trend towards video content instead of written sucks so much.
- samanthasu 2y ago
  same same
- nodamage 2y ago
  As far as I can tell the two main criticisms in the video are that: 1. The Redis API requires the developer to use different commands to retrieve/manipulate data depending on the type of data being stored. To retrieve a string you use GET, but if you want to retrieve a list it's LRANGE, for a set it's SMEMBERS, for a hash it's HGETALL. (As opposed to an API design which would allow you to call GET on all of the different data types and have it return the right thing.) 2. The lack of a predefined schema means you can overwrite values with different types. So you can create a list named "foo" and then overwrite it with a string named "foo" and then overwrite that with a hash named "foo" and Redis will happily do it, meaning the developer needs to keep track on their end what actual type any given key is holding onto. To me these criticisms come across as essentially saying "Redis doesn't behave like a RDBMS" to which I suppose antirez's point is "well, yeah, it's not supposed to".
  - tayo42 2y ago
    thanks for saving me some time!
apavlo 2y ago
> Wow, the reasons why Redis commands API suck in Andy's video (linked in the post) are the weakest ever. In my example, the API on a key changes based on its value type. And the same collection can have different value types mixed together. You've recreated the worst parts of IBM IMS from the 1960s. However, the original version of IMS only changed the API when a collection's backing data structure changed. Redis can change it on every key! We didn't get into the semantics of Redis' MULTI...EXEC, which the documentation mischaracterizes as "Transactions". I'm happy that at least you didn't use BEGIN...COMMIT.
- antirez 2y ago
  You totally miss that Redis is more like a remote interpreter with a DSL that manipulates data structures stored at global variables (keys): you (hopefully) would never complain about languages having this semantics. I don't think you understood how Redis collections work. The items are just strings, they can't be mixed like integers or strings together or whatever, nor collections can be nested. The Redis commands do type checking to ensure the application is performing the right operation. In your example, GET against a list, does not make sense because: 1. GET is the retrieve-the-key-of-string-type operation. 2. Having GET doing something like LRANGE 0 -1 would have many side effects. Getting for error a huge list and returning a huge data set without any reason, creating latency issues. Also having options for GET to provide ranges (SQL alike query languages horror story). And so forth. So each "verb" should do a specific action in a given data type. Are you absolutely sure you were exposed enough to the Redis API, how it works, and so forth? About MULTI/EXEC, when AOF with fsync configured correctly is used, MULTI/EXEC provide some of the transactional guarantees you think when you hear "transaction", but in general the concept refers to the fact that commands inside MULTI/EXEC have an atomic effect from the point of view of an external observer AND point-in-time RDB files (and AOF as well). MULTI / INCR a / INCR a / EXEC will always result in the observer to see either 2, 4, 6, 8, and so forth, and never 3 or 5. Anyway, I believe you didn't put enough efforts in understanding how really Redis works. Yet you criticized it with weak arguments in front of the most precious good we have: students. This is the sole reason why I wrote my first comment, I believe this to be a wrong teaching approach.
  - 2y ago
brightball 2y ago
Yea, I always use Redis for very specialized purposes. Like offloading a shared data structure between threads / processes / machines so that I don’t have to deal with thread safety issues.
- Spivak 2y ago
  I understand machines but threads?! Why introduce IPC overhead on the fastest/easiest way to share data? This is beyond a solved problem and your language probably has multiple ready-made battle tested solutions. In Python you don't even need a lib, dict is thread safe even in nogil.
  - pdhborges 2y ago
    In Python you don't even need a lib, dict is thread safe even in nogil. Is it? https://google.github.io/styleguide/pyguide.html#218-threading https://google.github.io/styleguide/pyguide.html#218-threadi...
    - Spivak 2y ago
      Yep! Not every operation you can do on a dict is thread safe but if you find a situation where it isn't it's a bug. https://github.com/python/cpython/issues/112075 https://github.com/python/cpython/issues/112075 Google is recommending people not rely on it because it does make dict subclasses not substitutable. It's easy enough to avoid the issue completely so in most cases you might as well do that.
  - brightball 2y ago
    Depends on the use case. Often if I’m sharing data between threads there’s a good chance I’ll want to scale it to another instance at some point so just going straight to Redis is pretty common. Especially if there’s a chance I’d want that data to persist across restarts. It’s one thing if I’m using a BEAM language, but otherwise I’ll usually reach for Redis.

nojito 2y ago

SQL is king and history has shown non-sql languages are not good which causes many non-sql DBMS's to adopt sql eventually.

antirez 2y ago
Many non-SQL DBs had query languages that were broken Javascript-ish versions of SQL. Of course, this is wrong, and people will eventually adopt SQL instead. But if your data model isn't anything like relational DBs, non-SQL makes a ton of sense. OP seems to miss exactly this, that the Redis query language is shaped on the Redis data model, that is basically alien to the relational model. The idea behind Redis data model is that "describe data" then "query those data in random ways" is conceptually nice but practically will not model very well many use cases. SQL databases plagued tech with performance issues for decades because of that. So Redis says instead: you need to put your data thinking about fundamental things like data structures and access times and the way you'll need those data back. And the API reflects this. You don't have to automatically agree with that. But you have to understand that, then provide your "I'm against" arguments. Especially if you are in front of young people listening to you.
- im_down_w_otp 2y ago
  Agreed. Many noSQL-boom-era databases eventually bolted on a SQL-esque layer, but that was also because they were mostly also all targeting "enterprise database" use cases and customers who both expected that and whose use cases largely fit with it. So, there was a lot of pressure to conform to norms when the advantage of not doing so wasn't immediately self-evident. We have a database [1] and query language [2] that's tailored to storing & querying trace/telemetry data produced by different layers and components of cyber-physical systems for systems engineers to analyze, verify, and validate what a complex system is doing. It's not quite a traditional relational problem. It's not quite a traditional time series problem. It's not quite a traditional graph problem. Addressing the way that systems engineers think about their domain in an effective way required coming up with something different. Are there caveats and rough edges? Sure. But, they're a lot less pernicious and onerous than the alternative of trying to leverage a bunch of ill-fitting menageries of different solutions. Redis is fit-for-purpose. So, it makes sense that its query interface would also express that. [1] https://docs.auxon.io/modality/ https://docs.auxon.io/modality/ [2] https://docs.auxon.io/speqtr/ https://docs.auxon.io/speqtr/
- nojito 2y ago
  >But if your data model isn't anything like relational DBs, non-SQL makes a ton of sense. OP seems to miss exactly this, that the Redis query language is shaped on the Redis data model, that is basically alien to the relational model. Sure...but all roads lead back to SQL eventually. Another recent example also mentioned in the OP is BigTable adopting SQL.
  - threeseed 2y ago
    > but all roads lead back to SQL eventually No it doesn't. SQL is designed for relational databases. For other forms i.e. JSON, Graph, Key/Value they all use other query languages.
  - physicles 2y ago
    If you write code that uses a hash map, would you insist on using SQL to query it? This makes no sense.
anovick 2y ago
SQL (and RDBMS in general) has its limitations, particularly with regards to recursive operations. An extended Datalog[1] can provide performance optimizations not available to RDBMS. [1]: https://dl.acm.org/doi/10.1145/3639271 https://dl.acm.org/doi/10.1145/3639271
KronisLV 2y ago
For what it's worth, SQL kind of sucks. It's just the de facto choice because it's extremely widespread and good enough for 80% of the use cases out there and what's missing can be kludged on top of it, either by specific DB vendors, or by various extensions. It's not too hard to come up with alternatives that improve upon individual aspects of SQL like https://prql-lang.org/ https://prql-lang.org/ but the barrier of entry is about as high as trying to make a huge social media network, most attempts will remain niche. Then again, most software kind of sucks, it's just that some of it also works. For example, the Linux FHS reads like an overcomplicated structure that is the way it is for historical reasons, but works in practice.

fforflo 2y ago

I've been working on something Redis-y over the holidays, and it has reinforced my view that it's the epitome of a 20%- 80% tool. I've always used the 20%, but anything beyond that sounds useless unless you've encountered the requirement in a production environment. The challenges Redis has been solving for years, never really touched the research/academic community (even the 20%). Even in the various taxonomies of DBS in the research literature, Redis was mentioned with a wave of the hand as an "in-memory" database, which undersells the important (for me) part of the "data structure" server. Putting the "database" after Redis could be a marketing misstep. Because it puts you in the is-it-sql territory. TL;DR: Redis is mostly appreciated by practitioners (web) developers. Academics find it lacking a theoretical foundation, so... meh.

Tanjreeve 2y ago
Developers know it's limits. Or you have developers with vague "scaling issues" or "buggy caching" who don't understand why they have them or suddenly start suffering from them at inconvenient moments.

nicoritschel 2y ago

With all due respect, the linked video was pretty fair. It didn't imply not to use Redis, just not as a primary datastore. I don't think folks work with Redis out of fondness for the model, but because it's the least worst datastore for caching, lightweight message broker, and simple realtime things like counters.

antirez 2y ago
Talking about the broken API argument here. Also Redis is particularly useful exactly in other situations compared to what OP says. Leaderboards style use cases with sorted sets are killer applications (super hard to model with SQL) of the data structure server thing. Apparently OP does not understand this and says "simple GET/SET" is what you should use Redis for. Redis has probabilistic data structures, the ability to implement complex queueing patterns, and so forth. That's where the value is. Otherwise we would still be just with Memcached without caring about Redis. Another killer app was Twitter initial use case (then they used it for pretty much everything): to cache latest N Tweets, using capped lists. I could continue forever. So OP argoment is flawed IMHO, for the above arguments, not fair. When you talk to students you need to make your homeworks. Really understand the system you are talking and provide a realistic image of it. Then, yes, if you want, criticize it as much as you want, with grounded arguments. You know what? I re-read this comment and it's embarassing I ever have to write this, because after 15 years of Redis history at such scale and popularity, pretty much everybody that was seriously exposed to Redis knows those stuff. Is tech culture really degraded so much that we have to restate the obvious? Do I really need to explain GET/SET is not exactly where Redis shines after 15 years of half the Internet used all the kind of Redis patterns?
- memhole 2y ago
  What are your thoughts about Rails switching to SQLite from Redis? I've only used Redis to store session data and cache app data. So my opinions are pretty limited and mostly positive. https://rubyonrails.org/2024/11/7/rails-8-no-paas-required https://rubyonrails.org/2024/11/7/rails-8-no-paas-required
  - 2y ago

tayo42 2y ago

> Andy's video (linked in the post) Is there a "to long didnt watch" summary any one knows of? I hate videos, but am curious lol

jghn 2y ago
Right there with you. The trend towards video content instead of written sucks so much.
samanthasu 2y ago
same same
nodamage 2y ago
As far as I can tell the two main criticisms in the video are that: 1. The Redis API requires the developer to use different commands to retrieve/manipulate data depending on the type of data being stored. To retrieve a string you use GET, but if you want to retrieve a list it's LRANGE, for a set it's SMEMBERS, for a hash it's HGETALL. (As opposed to an API design which would allow you to call GET on all of the different data types and have it return the right thing.) 2. The lack of a predefined schema means you can overwrite values with different types. So you can create a list named "foo" and then overwrite it with a string named "foo" and then overwrite that with a hash named "foo" and Redis will happily do it, meaning the developer needs to keep track on their end what actual type any given key is holding onto. To me these criticisms come across as essentially saying "Redis doesn't behave like a RDBMS" to which I suppose antirez's point is "well, yeah, it's not supposed to".
- tayo42 2y ago
  thanks for saving me some time!

apavlo 2y ago

> Wow, the reasons why Redis commands API suck in Andy's video (linked in the post) are the weakest ever. In my example, the API on a key changes based on its value type. And the same collection can have different value types mixed together. You've recreated the worst parts of IBM IMS from the 1960s. However, the original version of IMS only changed the API when a collection's backing data structure changed. Redis can change it on every key! We didn't get into the semantics of Redis' MULTI...EXEC, which the documentation mischaracterizes as "Transactions". I'm happy that at least you didn't use BEGIN...COMMIT.

antirez 2y ago
You totally miss that Redis is more like a remote interpreter with a DSL that manipulates data structures stored at global variables (keys): you (hopefully) would never complain about languages having this semantics. I don't think you understood how Redis collections work. The items are just strings, they can't be mixed like integers or strings together or whatever, nor collections can be nested. The Redis commands do type checking to ensure the application is performing the right operation. In your example, GET against a list, does not make sense because: 1. GET is the retrieve-the-key-of-string-type operation. 2. Having GET doing something like LRANGE 0 -1 would have many side effects. Getting for error a huge list and returning a huge data set without any reason, creating latency issues. Also having options for GET to provide ranges (SQL alike query languages horror story). And so forth. So each "verb" should do a specific action in a given data type. Are you absolutely sure you were exposed enough to the Redis API, how it works, and so forth? About MULTI/EXEC, when AOF with fsync configured correctly is used, MULTI/EXEC provide some of the transactional guarantees you think when you hear "transaction", but in general the concept refers to the fact that commands inside MULTI/EXEC have an atomic effect from the point of view of an external observer AND point-in-time RDB files (and AOF as well). MULTI / INCR a / INCR a / EXEC will always result in the observer to see either 2, 4, 6, 8, and so forth, and never 3 or 5. Anyway, I believe you didn't put enough efforts in understanding how really Redis works. Yet you criticized it with weak arguments in front of the most precious good we have: students. This is the sole reason why I wrote my first comment, I believe this to be a wrong teaching approach.
- 2y ago

brightball 2y ago

Yea, I always use Redis for very specialized purposes. Like offloading a shared data structure between threads / processes / machines so that I don’t have to deal with thread safety issues.

Spivak 2y ago
I understand machines but threads?! Why introduce IPC overhead on the fastest/easiest way to share data? This is beyond a solved problem and your language probably has multiple ready-made battle tested solutions. In Python you don't even need a lib, dict is thread safe even in nogil.
- pdhborges 2y ago
  In Python you don't even need a lib, dict is thread safe even in nogil. Is it? https://google.github.io/styleguide/pyguide.html#218-threading https://google.github.io/styleguide/pyguide.html#218-threadi...
  - Spivak 2y ago
    Yep! Not every operation you can do on a dict is thread safe but if you find a situation where it isn't it's a bug. https://github.com/python/cpython/issues/112075 https://github.com/python/cpython/issues/112075 Google is recommending people not rely on it because it does make dict subclasses not substitutable. It's easy enough to avoid the issue completely so in most cases you might as well do that.
- brightball 2y ago
  Depends on the use case. Often if I’m sharing data between threads there’s a good chance I’ll want to scale it to another instance at some point so just going straight to Redis is pretty common. Especially if there’s a chance I’d want that data to persist across restarts. It’s one thing if I’m using a BEAM language, but otherwise I’ll usually reach for Redis.