4 ms·

Redis is fast and great and I use it and love it a lot. That said, here are my top two: --- Redis is best for simple key -> value use case Redis is a key val

by bedobi 5y ago

Redis is fast and great and I use it and love it a lot. That said, here are my top two: --- Redis is best for simple key -> value use case Redis is a key value store, typically id -> entry, great for caching stuff etc. But if you're not ABSOLUTELY FING CERTAIN you will NEVER need anything more than that, as fast and great as Redis is, it's probably not the right tool for the job. Take it from someone who begrudgingly implemented a userId -> user store using Redis, pointing out that a regular db would be a better option as it's all but certain there will be a need for email -> user, username -> user etc etc indeces and then had to manually implement those indeces into the same Redis when, lo and behold, they became needed gaaah --- Redis often isn't meaningfully faster than MySQL, even for key -> value lookup An AWS MySQL RDS that is sized to contain the entire db in memory and has the correct settings will also be de facto fully in memory and just as fast as Redis but now you're getting a whole relational db with all the added benefit that entails it will also be vastly easier to grow as required (unless anything has changed since I last did this, if you reach the limit of your Redis instance, you have to create a one with more capacity and manually migrate all the entries (including real time changes) into the new one which is a huge pain)

hardwaresofton 5y ago
As someone who recently migrated a project off of redis-as-a-main-store, please use a real database. I found that Postgres could do the same requests in about 2x-7x the time depending on what the requests were and this was without heavy optimization (if you limit the object sizes you could actually get away with covering indices which would probably get even closer). Also, KeyDB[0] looks like quite the project. I used it when testing backups and some other stuff and it was quite easy to use, scaled to the compute well, is able to easily use flash (SSD/NVMe), pretty nice. [0]: keydb.dev
sateesh 5y ago
An AWS MySQL RDS that is sized to contain the entire db in memory and has the correct settings will also be defacto fully in memory and just as fast as Redis but now you're getting a whole relational db with all the added benefit that entails Are you making this assertion based on your practical experience or making a theoretical point. If it is the latter then probably it might not be correct. This is what the book Designing Data Intensive Applications from Martin Kleppman has to say: Counterintuitively, the performance advantage of in-memory databases is not due to the fact that they don't need to read from disk. Even a disk-based storage engine may never need to read from disk if you have enough memory, because the operating system caches recently used disk blocks in memory anyway. Rather, they can be faster because they can avoid the overheads of encoding in-memory data structures in a form that can be written to disk. (Chapter-03 Storage and Retrieval) So it seems even the usage as you described the DB lookups won't be faster than Redis lookup. I haven't fully understood the reasoning author provides (any pointers providing more explanation are welcome). The book provides the paper https://dl.acm.org/doi/10.1145/1376616.1376713 https://dl.acm.org/doi/10.1145/1376616.1376713 (OLTP Through the Looking Glass, and What We Found There") as the basis for the above assertion.
- kitd 5y ago
  Even a disk-based storage engine may never need to read from disk if you have enough memory, because the operating system caches recently used disk blocks in memory anyway. OT, but this is the premise behind Kafka's log storage as well. Which is why it's great for sequential reads from the front of the log, and not as a general-purpose DB, which too many people want it to be.
- bedobi 5y ago
  Practical experience! Traditional db def wont be faster than Redis. (at least not in my experience) But what I'm saying is traditional db also often won't be meaningfully slower than Redis either, even for key -> value lookup where Redis shines. Overwhelmingly, when I need to store and retrieve anything, I reach for traditional dbs first by default, unless there's good reason to use some other technology. If and when load starts to become a problem, tweaking the settings, adding more read replicas etc is often enough. But if it's not, it's always perfectly possible and simple enough to add Redis on top. I feel like that gives you the most bang for your buck and the best of both worlds. But YMMV!

by bedobi 5y ago

hardwaresofton 5y ago

As someone who recently migrated a project off of redis-as-a-main-store, please use a real database. I found that Postgres could do the same requests in about 2x-7x the time depending on what the requests were and this was without heavy optimization (if you limit the object sizes you could actually get away with covering indices which would probably get even closer). Also, KeyDB[0] looks like quite the project. I used it when testing backups and some other stuff and it was quite easy to use, scaled to the compute well, is able to easily use flash (SSD/NVMe), pretty nice. [0]: keydb.dev

sateesh 5y ago

An AWS MySQL RDS that is sized to contain the entire db in memory and has the correct settings will also be defacto fully in memory and just as fast as Redis but now you're getting a whole relational db with all the added benefit that entails Are you making this assertion based on your practical experience or making a theoretical point. If it is the latter then probably it might not be correct. This is what the book Designing Data Intensive Applications from Martin Kleppman has to say: Counterintuitively, the performance advantage of in-memory databases is not due to the fact that they don't need to read from disk. Even a disk-based storage engine may never need to read from disk if you have enough memory, because the operating system caches recently used disk blocks in memory anyway. Rather, they can be faster because they can avoid the overheads of encoding in-memory data structures in a form that can be written to disk. (Chapter-03 Storage and Retrieval) So it seems even the usage as you described the DB lookups won't be faster than Redis lookup. I haven't fully understood the reasoning author provides (any pointers providing more explanation are welcome). The book provides the paper https://dl.acm.org/doi/10.1145/1376616.1376713 https://dl.acm.org/doi/10.1145/1376616.1376713 (OLTP Through the Looking Glass, and What We Found There") as the basis for the above assertion.

kitd 5y ago
Even a disk-based storage engine may never need to read from disk if you have enough memory, because the operating system caches recently used disk blocks in memory anyway. OT, but this is the premise behind Kafka's log storage as well. Which is why it's great for sequential reads from the front of the log, and not as a general-purpose DB, which too many people want it to be.
bedobi 5y ago
Practical experience! Traditional db def wont be faster than Redis. (at least not in my experience) But what I'm saying is traditional db also often won't be meaningfully slower than Redis either, even for key -> value lookup where Redis shines. Overwhelmingly, when I need to store and retrieve anything, I reach for traditional dbs first by default, unless there's good reason to use some other technology. If and when load starts to become a problem, tweaking the settings, adding more read replicas etc is often enough. But if it's not, it's always perfectly possible and simple enough to add Redis on top. I feel like that gives you the most bang for your buck and the best of both worlds. But YMMV!