16 ms·

You might not need Redis

241 points by jcartw 2y ago

evidencetamper 2y ago
> Redis is arguably the most well regarded tech of all. REDIS tradeoffs have been so widely discussed because many, many engineers disagree with them. REDIS is so lowly regarded that some companies ban it completely, making their engineers choose between memcached or something more enterprisey (hazelcast these days, Coherence some time ago).
- deleted 2y ago
  [deleted]
- throwaway7783 2y ago
  Why is Redis regarded so lowly? Now, Redis Search, I understand. Very buggy, search index misses records and doesn't tell that it missed them and so on. Core Redis is a solid piece of technology, afaik
  - Joel_Mckay 2y ago
    Probably the same reasons dynamically-scaled Aerospike is not as popular. Redis primarily has a reputation of just "caching" stuff in memory, but probably the rates of CVE in the distributed clients soured some peoples opinions. Distributed systems is hard, and what seems like a trivial choice about design constraints can quickly balloon in cost. =3
- do_not_redeem 2y ago
  This comment has a lot of words but says nothing. "Many engineers" disagree with every piece of software, and "some companies" ban every piece of software. Why are you writing Redis in all caps like it's an acronym? Reminds me of those old dog C programmers who write long rants about RUST with inaccuracies that belie the fact they've never actually used it.
  - Xiol32 2y ago
    Same vibes as people who write SystemD.
  - karmakaze 2y ago
    If anything it's ReDiS, Remote Dictionary Server. Also pronounced to sound like 'remote', unlike the color of its logo (which would be spelled Reddis).
- virtue3 2y ago
  Redis is probably one of the most well written pieces of software I’ve ever looked at. The source code is magnificent.
  - pphysch 2y ago
    The original maintainer went on to write a sci-fi novel
  - foobarian 2y ago
    I second this, specifically the geohash code is a work of art.
- tayo42 2y ago
  I work at a company that banned redis after an outage. kind of blunt, it feels like such an amateur decision to have made. So reactionary instead of trying to learn it. It's large popular internet company too
  - secondcoming 2y ago
    Was it in the early days of redis? I have a dedicated server that's running an instance of redis on each of the 32 cores and it's been up for over a year now. Each core is doing about 30k QPS
sorokod 2y ago
So premature optimization causes Redis?
- deleted 2y ago
  [deleted]
dinobones 2y ago
It’s not about Redis vs not Redis, it’s about working with data that does not serialize well or lend itself well to extremely high update velocity. Things like: counters, news feeds, chat messages, etc The cost of delivery for doing these things well with a LSM based DB or RDB might actually be higher than Redis. Meaning: you would need more CPUs/memory to deliver this functionality, at scale, than you would with Redis, because of all the overhead of the underlying DB engine. But for 99% of places that aren’t FAANG, that is fine actually. Anything under like 10k QPS and you can do it in MySQL in the dumbest way possible and no one would ever notice.
- packetlost 2y ago
  I've only ever seen Redis used in two scenarios: storing ephemeral cache data to horizontally scale Django applications and for ephemeral job processing where the metadata about the job was worthless. I reevaluated it for a job processing context a couple of years ago and opted for websockets instead because what I really needed was something that outlived an HTTP timeout. I've never actually seen it used in a case where it wasn't an architecture smell. The codebase itself is pretty clean and the ideas it has are good, but the idea of externalizing datastructures like that just doesn't seem that useful if you're building something correctly.
  - pdimitar 2y ago
    To your last: yep, especially having in mind that Redis is ephemeral. I've had much more success with SQLite + a bunch of stricter validators (as SQLite itself is sadly pretty loose), and more performance too.
  - ljm 2y ago
    Redid + Sidekiq was a default for a long time in the Rails world as well, but it’s an unnecessary complication (and expense) for most use cases. Just use your existing DB until you need to seriously scale up, and then look at a message queue. I’ve used Redis for leaderboards and random matchmaking though, stuff which is doable in postgres but is seriously write-heavy and a bit of a faff. Gives you exactly the sort of goodies you need on top of a K/V store without being difficult to set up. As for caching - it’s nice to use as an engineer for sure, but pretty pricey. It wouldn’t be my default choice any more.
dorianniemiec 2y ago
I have found that for websites that use Joomla, file cache is faster than Redis cache, at least for me.
- lucb1e 2y ago
  What data is cached in there? And you're almost certainly still caching it to RAM, not sure if you're aware that the kernel keeps in-memory copies of recently-read/written pieces of files
mannyv 2y ago
Caching is a funny thing. Just like anything you need to understand if and why you need it. And one other thing is you should be able to fall back if your cache is invalidated. In our case we keep a bunch of metadata in redis that’s relatively expensive (in cost and speed) to pull/walk in realtime. And it needs to be fast and support lots of clients. The latter sort of obviates direct-to-database options.
bassp 2y ago
I agree with the author 100% (the TanTan anecdote is great, super clever work!), but.... sometimes you do need Redis, because Redis is the only production-ready "data structure server" I'm aware of If you want to access a bloom filter, cuckoo filter, list, set, bitmap, etc... from multiple instances of the same service, Redis (slash valkey, memorydb, etc...) is really your only option
- jasonthorsness 2y ago
  Yes, while the default idea of Redis might be to consider it a key/value cache, the view of the project itself is definitely about being a "data structure server" - it's right at the top of the https://github.com/redis/redis/blob/unstable/README.md https://github.com/redis/redis/blob/unstable/README.md and antirez has focused on that (I can't find one quote I am looking for specifically but it's evident for example in discussion on streams https://antirez.com/news/114 https://antirez.com/news/114). Although I've definitely seen it be used just as a key/value store in the deployments I'm familiar with ¯\_(ツ)_/¯
- e_hup 2y ago
  All of those can be serialized and stored in an RDMS. You don't need Redis for that.
  - bassp 2y ago
    They can (and that's probably the right choice for a lot of use cases, especially for small data structures and infrequently updated ones), but serializing and storing them in a database requires you to (in your application code) implement synchronization logic and pay the performance cost for said logic; for instance, if you want to `append` to a shared list, you need to deserialize the list, append to the end of it in your application code, and write it back to the DB. You'd need use some form of locking to prevent appends from overwriting each other, incurring a pretty hefty perf penalty for hot lists. Also, reading an entire list/tree/set/whatever back just to add/delete one element is very wasteful (bandwidth/[de]serialization cost-wise)
    - 2y ago
tombert 2y ago
I have gotten in arguments with people who over-deploy Redis. Redis is cool, I don't dislike it or anything, but a lot of the time when people use it, it actually slows things down. Using it, you're introducing network latency and serialization overhead. Sometimes that's worth it, especially if your database is falling over, but a lot of the time people use it and it just makes everything more complex and worse. If you need to share cached data across processes or nodes, sometimes you have to use it, but a lot of the stuff I work with is partitioned anyway. If your data is already partitioned, you know what works well a lot of the time? A boring, regular hashmap. Pretty much every language has some thread-safe hashmap in there, and a lot of them have pretty decent libraries to handle invalidation and expiration if you need those. In Java, for example, you have ConcurrentHashMap for simple stuff, and Guava Caches or Caffeine Caches for more advanced stuff. Even the slowest [1] local caching implementation will almost certainly be faster than anything that hits the network; in my own testing [2] Caffeine caches have sub-microsecond `put` times, and you don't pay any serialization or deserialization cost. I don't think you're likely to get much better than maybe sub-millisecond times with Redis, even in the same data center, not to mention if you're caching locally that's one less service that you have to babysit. Again, I don't hate Redis, there are absolutely cases where it's a good fit, I just think it's overused. [1] Realistic I mean, obviously any of use could artificially construct something that is slow as we want. [2] https://blog.tombert.com/posts/2025-03-06-microbenchmark-errata/ https://blog.tombert.com/posts/2025-03-06-microbenchmark-err... This is my own blog, feel free to not click it. Not trying to plug myself, just citing my data.
- hajimuz 2y ago
  In most cases It’s not about the speed, it’s about data sharing for containers or distributed systems. Filesystem or in-memory doesn’t work. I agree that in most cases a normal database is enough though.
  - tombert 2y ago
    Yeah I mentioned that, if you need to share stuff between processes or differnet nodes, then maybe Redis might be a fit. But I've seen people use Redis as a glorified "global variable" for stuff like Kafka streaming. The data is already partitioned, it's not going to be used across multiple nodes, and now you've introduced another service to look at and made everything slower because of the network. A global hashmap (or cache library, like previously mentioned) would do the job faster, with less overhead, and the code would be simpler.
jayd16 2y ago
I agree that Redis can be an architectural smell. You can shoehorn it in where it doesn't belong. It's easy to work with and does it's job so well that for a certain type of dev it's easier to implement caching than understand enough SQL to add proper indices.
- tengbretson 2y ago
  Its funny, because once you go through the effort of determining which query fields you should compose to make your cache key, you're 95% of the way to the index you should have had in the first place.
bdcravens 2y ago
Another category is using Redis indirectly via dependencies. For example in Rails, Sidekiq is a common background job library. However, there are now Postgresql-backed options (like GoodJob and the baked-in Solid Queue, which supports other RDBMSes as well)
- jherdman 2y ago
  I just wanted to take a moment to highlight both GoodJob and Solid Queue. They're excellent choices, and any RoR folks reading these comments should give them a fair shake.
sgarland 2y ago
> 'Why can't we just store this data on the shards in PostgreSQL, next to the swipes?'. The data itself would be microscopic in comparison and the additional load would also be microscopic in comparison to what these servers were already doing. I'm assuming based on the rest of the article that the author and team knew what they were doing, but if you aren't familiar with Postgres' UPDATE strategy and HOT updates [0], you should familiarize yourself before attempting this, otherwise, you're going to generate a massive amount of WAL traffic and dead tuples at scale. [0]: https://www.postgresql.org/docs/current/storage-hot.html https://www.postgresql.org/docs/current/storage-hot.html
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
igortg 2y ago
I followed with this rationale in a small project and opted for PostgreSQL pub/sub instead Redis. But I went through so much trouble correctly handling PostgreSQL disconnections that I wonder if Redis wouldn't be the better choice.
phendrenad2 2y ago
I like to run a single redis server process on each instance that's serving the webapp, and also have a shared redis cluster for shared cached state (like user sessions). That way there's a fast local cache for people to use for caching API calls and other things where duplication is preferable to latency.
- tengbretson 2y ago
  If it's purely ephemeral and not replicated between instances what are you getting from redis that couldn't be done with SQLite or even a good old hashmap in a separate thread?
  - phendrenad2 2y ago
    Redis is probably slightly faster than sqlite, and developers are used to using redis primitives and APIs.
  - ricardobeat 2y ago
    You get standardized data structures and consistent operations on them for free.
kflgkans 2y ago
You might not need a cache. In my previous company (~7 years) all teams around me were introducing caches left and right and getting into a lot of complexity and bugs. I persevered and always pushed back adding caches to apps in my team. Instead focusing on improving the architecture and seeking other performance improvements. I can proudly say my teams have stayed cached-free for those 7 years.
xyzzy9563 2y ago
Redis is useful if you have multiple backend servers that need to synchronize state. If you don’t have this problem then improving your caching I.e. using Cloudflare is probably the best approach, along with local caching on a single server.
g9yuayon 2y ago
When I was in Uber back in 2015, my org was trying to convert zip-code-based geo partitioning with a hexagon-based scheme. Instead of partitioning a city into on average tens of zip codes, we may partition the city into potentially hundreds of thousands of hexagons and dynamically create areas. The first launch was in Phoenix, and the team who was responsible for the launch stayed up all night for days because they could barely scale our demand-pricing systems. And then the global launch of the feature was delayed first by days, then by weeks, and then by months. It turned out Uber engineers just loved Redis. Having a need to distribute your work? Throw that to Redis. I remember debating with some infra engineers why we couldn't throw in more redis/memcached nodes to scale our telemetry system, but I digressed. So, the price service we built was based on Redis. The service fanned out millions of requests per second to redis clusters to get information about individual hexagons of a given city, and then computed dynamic areas. We would need dozens of servers just to compute for a single city. I forgot the exact number, but let's say it was 40 servers per an average-sized city. Now multiply that by the 200+ cities we had. It was just prohibitively expensive, let alone that there couldn't other scalability bottlenecks for managing such scale. The solution was actually pretty simple. I took a look at the algorithms we used, and it was really just that we needed to compute multiple overlapping shapes. So, I wrote an algorithm that used work-stealing to compute the shapes in parallel per city on a single machine, and used Elasticsearch to retrieve hexagons by a number of attributes -- it was actually a perfect use case for a search engine because the retrieval requires boolean queries of multiple attributes. The rationale was pretty simple too: we needed to compute repetitively on the same set of data, so we should retrieve the data only once for multiple computations. The algorithm was of merely dozens of lines, and was implemented and deployed to production over the weekend by this amazing engineer Isaac, who happens to be the author of the library H3. As a result, we were able to compute dynamic areas for 40 cities, give or take, on a single machine, and the launch was unblocked.
- muggermuch 2y ago
  Cool anecdote, thanks for sharing!
- pstuart 2y ago
  The aforementioned hex mapping tool: https://h3geo.org/ https://h3geo.org/ For anyone doing geo queries it's a powerful tool.
lukaslalinsky 2y ago
For years I've tried to use Redis as a persistent data store. I've only been dissatisfied, having bad experiences with both sentinel and cluster. Most of my service outages were linked to Redis replication getting broken. Then I decided to give up and use it only as an empehemral cache. I have a large number of standalone Redis instances (actually, now they are Valkey), no storage, only memory, and have Envoy proxy on top of them for monitoring and sharding. And I'm really happy with it, storing hundreds of GBs of data there, if one goes down, only a small part of the data needs to be reloaded from the primary source, and with the Envoy proxy, applications see it as a single Redis server. I was considering just replacing it with memcached, but the redis data model is more rich, so I kept using it, just not expecting anything put there to be actually stored forever.
- anonymousDan 2y ago
  What were the problems you had when using it as a persistent data store if you don't mind me asking? Also, what kind of workloads did you have?
  - lukaslalinsky 2y ago
    The problematic use case was using Redis as a counter for various app related metrics, which I want to make visible to the customer later on. App was writing to Redis, background job was reading these partially aggregated metrics from Redis and writing them to another database. So you could say it was a remote buffer. On way too many occasions, after short failure of the host where the current master instance was running, the replica was not propagated to master. That was using redis sentinel. I guess my main issue was always redis sentinel.
progbits 2y ago
Redis as ephemeral cache is ok, but nothing extra. Redis as transactional, distributed and/or durable storage is pretty poor. Their "active active" docs on conflict resolution for example don't fill me with confidence given there is no formalism, just vague examples. But this comes from people who not only not know how do distributed locks, they refuse to learn when issues are pointed out to them: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html https://martin.kleppmann.com/2016/02/08/how-to-do-distribute... Every time I find code that claims to do something transactional in Redis which is critical for correctness, not just latency optimization, I get worried.
ent101 2y ago
Shameless plug for my project: https://github.com/HeyPuter/kv.js https://github.com/HeyPuter/kv.js Don't kill me :')
- 0xblinq 2y ago
  That's a great project! I'm definitely going to give it a try.
vvpan 2y ago
Really the most useful feature in Redis to me is TTL - so easy to cache things and not worry about them being too stale. The second most useful is that you can attach something like BullMQ to it and get cron/scheduling for a couple lines of code.
superq 2y ago
The issues that I have with Redis are not at all its API (which is elegant and brilliant) or even its serialized, single-core, single-threaded design, but its operational hazards. As a cache or ephemeral store like a throttling/rate limiting, lookup tables, or perhaps even sessions store, it's great; but it's impossible to rely on the persistence options (RDB, AOF) for production data stores. You usually only see this tendency with junior devs, though. It might be a case where "when all you have is a hammer, all you see are nails", or when someone discovers Redis (or during the MongoDB hype cycle ten years ago), which seems like it's in perfect alignment with their language datatypes, but perhaps this is mostly because junior devs don't have as many production-ready databases (from SQL like Postgresql, CockroachDB, Yugabyte to New/NoSQL like ScyllaDB, YDB, Aerospike) to fall back on. Redis shines as a cache for small data values (probably switch to memcache for larger values, which is simpler key-value but generally 3 to 10 times faster for that more narrow use case, although keep an eye on memory fragmentation and slab allocation) Just think carefully before storing long-term data in it. Maybe don't store your billing database in it :)
- noisy_boy 2y ago
  I have seen horrifying use of Redis where I inherited the maintainance of an application whose original developer implemented his own home grown design to manage relationships between different types of key value pairs, pretending like they were tables including cross-referencing logic; it took me a week to just add test cases with sufficient logging to reveal the "schema" and mutation logic. All this with the non-technical manager wondering why it took so long to make the change which directly dependended on understanding this. To top it all, the code was barely better than spaghetti with less than ten lines of comments across maybe 5k LOC. The irony was that this was not a latency sensitive application - it did data quality checks and could have been implemented in a much more cleaner and flexible way using, e.g., PostgreSQL.
  - intelVISA 2y ago
    Non-technical 'manager'
0xbadcafebee 2y ago
Software development today is largely just people repeating what other people do without thinking. Which is how human culture works; we just copy what everyone else is doing, because it's easier, and that becomes "normal", whatever it is. In the software world in the mid 00's, the trend started to work around the latency, cost and complexity of expensive servers and difficult databases by relying on the speed of modern networks and RAM. This started with Memcached and moved on to other solutions like Redis. (this later evolved into NoSQL, when developers imagined that simply doing away with the complexity of databases would somehow magically remove their applications' need to do complex things... which of course it didn't, it's the same application, needing to do a complex thing, so it needs a complex solution. computers aren't magic. we have thankfully passed the hype cycle of NoSQL, and moved on to... the hype cycle for SQLite) But the tradeoff was always working around one limitation by adding another limitation. Specifically it was avoiding the cost of big databases and the expertise to manage them, and accepting the cost of dealing with more complex cache control. Fast forward to 2025 and databases are faster (but not a ton faster) and cheaper (but not a ton cheaper) and still have many of the same limitations (because dramatically reinventing the database would have been hard and boring, and no software developer wants to do hard and boring things, when they can do hard and fun things, or ignore the hard things with cheap hacks and pretend there is no consequence to that). So people today just throw a cache in between the database, because 1) databases are still kind of stupid and hard (very very useful, but still stupid and hard) and 2) the problems of cache complexity can be ignored for a while, and putting off something hard/annoying/boring until later is a human's favorite thing. No, you don't need Redis. Nobody needs Redis. It's a hack to avoid dealing with stateless applications using slow queries on an un-optimized database with no fast read replicas and connection limits. But that's normal now.
- briandear 2y ago
  Recently just left a MongoDB project. A total nightmare.
- edoceo 2y ago
  > hype cycle for SQLite Drop Redis, replace with in-memory SQLite. But for real, the :memory: feature is actually pretty awesome!
khaki54 2y ago
I was reading an article last week where the author did a bunch of testing and found Redis is only 3-5x faster than just using a multi model NoSQL. Basically it rarely makes sense to use Redis. https://adamfowler.org/2015/09/07/hybrid-nosql-marklogic-as-a-k-v-store/ https://adamfowler.org/2015/09/07/hybrid-nosql-marklogic-as-...
- dcmatt 2y ago
  "only 3-5x faster... rarely makes sense to use Redis" This is an illogical conclusion. 3-5x faster is substantial!
  - lucb1e 2y ago
    I agree, although it also depends on the context. If a page needs to fetch lots of data and takes 3 full seconds to load, the middle case (4x improvement) makes it go from an amount of time where you glance at your messages to on par with the average javascript-laden website we're used to anyway (except in this case it's legitimate loading time). If, however, we're talking about most pages, that are mostly static and just fetch some article contents and author name from the DB in 2 milliseconds... yeah, it can be 10x faster and nobody cares
harrall 2y ago
Two things: I think people forget (or don’t know) that adding data storage system to your architecture also involves management, scaling, retention, and backup. It’s not free. And second, sometimes you do need to invest in storage to permit failover or to minimize latency but people do it for traffic when they really have little traffic. A server from 20 years ago could push a lot of traffic and systems have gotten only beefier.
antirez 2y ago
I believe that the issue is that the culture about Redis usage didn't evolve as much as its popularity. To use it memcached alike has many legitimate use cases but it's a very reductive way to use it. For instance sorted set ranking is something that totally changes the dynamics of what you can and can't do with traditional databases. Similarly large bitmaps that allow to retain very fast real time one bit information to do analytics otherwise very hard to do is another example. Basically, Redis helps a lot more as the company culture around it increases, more patterns are learned, and so forth. But in this regard a failure on the Redis (and my) side is that there isn't a patterns collection book: interviewing folks that handled important use cases (think at Twitter) to understand the wins and the exact usage details and data structure usages. Even just learning the writable cache pattern totally changes the dynamics of your Redis experience.
- kshitij_libra 2y ago
  Do you plan to write the book ? I’d like to read it
  - mleonhard 2y ago
    Yes, please write it. I will buy and read it.
    - jordiburgos 2y ago
      I'll buy it, read it and take notes.
notjoemama 2y ago
> The first usage was to cache the result of an external API call to lookup geolocation of users, so that the service in question could process location requests quicker, and also run cheaper. BAAAHHH HAAHA HA HA HA HHHAA. Ok, look. Just because it's in Redis does not disqualify the clause in the geo service's license that you NOT permanently store the data. The author did not say that's what they were doing but a very similar thing came up in a previous work place for me and we chose to call the service for every lookup the way the service company expected their customers to do. I recall the suggestion in using a caching database as a workaround and it was not made by the engineering team. Sorry, I'm still chuckling at this one...
- inib 2y ago
  We use Redis to store geocoding data from Google Geocoding API with a TTL of 30 days, per the policy. It's $4/1000 requests. There'd no business without the cache.
- dimgl 2y ago
  I don't understand this comment. Without caching, you'd be at the mercy of the geolocation company. I've deployed a service that does exactly this.
rc_mob 2y ago
Yeah I just removed redis from my php app. It runs slightly slower but not enough that my user would notice. Redis was overkill
simonw 2y ago
If you work at a company where teams keep on turning to Redis for different features, there's a chance that it's an indication that the process for creating new database tables in your relational store has too much friction!
alberth 2y ago
> A single beefy Redis (many cores, lots of RAM type of machine) should be able to handle the load I thought Redis was single threaded running on a single core. Having multiple cores provides no benefit (and arguably could hurt since large multicore systems typically have a lower clock)
- sriku 2y ago
  Redis itself, yes. However there are Redis compatible tools that go beyond single core - KeyDB, DragonFly and ARDB overcome some limitations.
cyberax 2y ago
Redis (or Valkey) has a bunch of functionality other than caching. It has a very useful event streaming system and task queueing system. You can use Postgres instead of them, but then it can easily become a bottleneck.
alberth 2y ago
This post seems less about Redis, and more broadly about - why you should never introduce new technologies into your existing stack (only do so when your existing stack can’t keep up).
briandear 2y ago
Rails adding SolidCache and SolidQueue (or Good Job) was what made me not even think of Redis anymore in the rails world.
dimgl 2y ago
I'm really surprised that the pendulum has swung so far in the other direction that people are recommending not to use Redis. Sure, don't introduce a data store into your stack unless you need it. But if you had to introduce one, Redis still seems like one of the best to introduce? It has fantastic data structures (like sorted sets, hash maps), great performance, robust key expiry, low communication overhead, low runtime overhead... I mean, the list goes on.
- esafak 2y ago
  I'd like to draw attention to its probabilistic data structures, in particular: HyperLogLog, Bloom filter, Cuckoo filter, t-digest, Top-K, Count-min sketch https://redis.io/docs/latest/develop/data-types/probabilistic/ https://redis.io/docs/latest/develop/data-types/probabilisti...
  - dimgl 2y ago
    This is wild, I had no idea it even had this.
- ks2048 2y ago
  Yeah, my first thought was, “I don’t need redis, but I want redis”.
- hot_gril 2y ago
  Or a simple KV cache, which is pretty often something you need. Sure you can use Postgres for this if you really don't want another dep, but Redis or Memcached is better suited for it and probably cheaper.
high_na_euv 2y ago
Case 1 Why not store this value on client side and sync it from server from time to time? Or it wasnt used by client?
deleted 2y ago
[deleted]
hackburg 2y ago
[dead]

241 points by jcartw 2y ago

> Redis is arguably the most well regarded tech of all. REDIS tradeoffs have been so widely discussed because many, many engineers disagree with them. REDIS is so lowly regarded that some companies ban it completely, making their engineers choose between memcached or something more enterprisey (hazelcast these days, Coherence some time ago).

deleted 2y ago
[deleted]
throwaway7783 2y ago
Why is Redis regarded so lowly? Now, Redis Search, I understand. Very buggy, search index misses records and doesn't tell that it missed them and so on. Core Redis is a solid piece of technology, afaik
- Joel_Mckay 2y ago
  Probably the same reasons dynamically-scaled Aerospike is not as popular. Redis primarily has a reputation of just "caching" stuff in memory, but probably the rates of CVE in the distributed clients soured some peoples opinions. Distributed systems is hard, and what seems like a trivial choice about design constraints can quickly balloon in cost. =3
do_not_redeem 2y ago
This comment has a lot of words but says nothing. "Many engineers" disagree with every piece of software, and "some companies" ban every piece of software. Why are you writing Redis in all caps like it's an acronym? Reminds me of those old dog C programmers who write long rants about RUST with inaccuracies that belie the fact they've never actually used it.
- Xiol32 2y ago
  Same vibes as people who write SystemD.
- karmakaze 2y ago
  If anything it's ReDiS, Remote Dictionary Server. Also pronounced to sound like 'remote', unlike the color of its logo (which would be spelled Reddis).
virtue3 2y ago
Redis is probably one of the most well written pieces of software I’ve ever looked at. The source code is magnificent.
- pphysch 2y ago
  The original maintainer went on to write a sci-fi novel
- foobarian 2y ago
  I second this, specifically the geohash code is a work of art.
tayo42 2y ago
I work at a company that banned redis after an outage. kind of blunt, it feels like such an amateur decision to have made. So reactionary instead of trying to learn it. It's large popular internet company too
- secondcoming 2y ago
  Was it in the early days of redis? I have a dedicated server that's running an instance of redis on each of the 32 cores and it's been up for over a year now. Each core is doing about 30k QPS

sorokod 2y ago

So premature optimization causes Redis?

deleted 2y ago
[deleted]

dinobones 2y ago

It’s not about Redis vs not Redis, it’s about working with data that does not serialize well or lend itself well to extremely high update velocity. Things like: counters, news feeds, chat messages, etc The cost of delivery for doing these things well with a LSM based DB or RDB might actually be higher than Redis. Meaning: you would need more CPUs/memory to deliver this functionality, at scale, than you would with Redis, because of all the overhead of the underlying DB engine. But for 99% of places that aren’t FAANG, that is fine actually. Anything under like 10k QPS and you can do it in MySQL in the dumbest way possible and no one would ever notice.

packetlost 2y ago
I've only ever seen Redis used in two scenarios: storing ephemeral cache data to horizontally scale Django applications and for ephemeral job processing where the metadata about the job was worthless. I reevaluated it for a job processing context a couple of years ago and opted for websockets instead because what I really needed was something that outlived an HTTP timeout. I've never actually seen it used in a case where it wasn't an architecture smell. The codebase itself is pretty clean and the ideas it has are good, but the idea of externalizing datastructures like that just doesn't seem that useful if you're building something correctly.
- pdimitar 2y ago
  To your last: yep, especially having in mind that Redis is ephemeral. I've had much more success with SQLite + a bunch of stricter validators (as SQLite itself is sadly pretty loose), and more performance too.
- ljm 2y ago
  Redid + Sidekiq was a default for a long time in the Rails world as well, but it’s an unnecessary complication (and expense) for most use cases. Just use your existing DB until you need to seriously scale up, and then look at a message queue. I’ve used Redis for leaderboards and random matchmaking though, stuff which is doable in postgres but is seriously write-heavy and a bit of a faff. Gives you exactly the sort of goodies you need on top of a K/V store without being difficult to set up. As for caching - it’s nice to use as an engineer for sure, but pretty pricey. It wouldn’t be my default choice any more.

dorianniemiec 2y ago

I have found that for websites that use Joomla, file cache is faster than Redis cache, at least for me.

lucb1e 2y ago
What data is cached in there? And you're almost certainly still caching it to RAM, not sure if you're aware that the kernel keeps in-memory copies of recently-read/written pieces of files

mannyv 2y ago

Caching is a funny thing. Just like anything you need to understand if and why you need it. And one other thing is you should be able to fall back if your cache is invalidated. In our case we keep a bunch of metadata in redis that’s relatively expensive (in cost and speed) to pull/walk in realtime. And it needs to be fast and support lots of clients. The latter sort of obviates direct-to-database options.

bassp 2y ago

I agree with the author 100% (the TanTan anecdote is great, super clever work!), but.... sometimes you do need Redis, because Redis is the only production-ready "data structure server" I'm aware of If you want to access a bloom filter, cuckoo filter, list, set, bitmap, etc... from multiple instances of the same service, Redis (slash valkey, memorydb, etc...) is really your only option

jasonthorsness 2y ago
Yes, while the default idea of Redis might be to consider it a key/value cache, the view of the project itself is definitely about being a "data structure server" - it's right at the top of the https://github.com/redis/redis/blob/unstable/README.md https://github.com/redis/redis/blob/unstable/README.md and antirez has focused on that (I can't find one quote I am looking for specifically but it's evident for example in discussion on streams https://antirez.com/news/114 https://antirez.com/news/114). Although I've definitely seen it be used just as a key/value store in the deployments I'm familiar with ¯\_(ツ)_/¯
e_hup 2y ago
All of those can be serialized and stored in an RDMS. You don't need Redis for that.
- bassp 2y ago
  They can (and that's probably the right choice for a lot of use cases, especially for small data structures and infrequently updated ones), but serializing and storing them in a database requires you to (in your application code) implement synchronization logic and pay the performance cost for said logic; for instance, if you want to `append` to a shared list, you need to deserialize the list, append to the end of it in your application code, and write it back to the DB. You'd need use some form of locking to prevent appends from overwriting each other, incurring a pretty hefty perf penalty for hot lists. Also, reading an entire list/tree/set/whatever back just to add/delete one element is very wasteful (bandwidth/[de]serialization cost-wise)
  - 2y ago

tombert 2y ago

I have gotten in arguments with people who over-deploy Redis. Redis is cool, I don't dislike it or anything, but a lot of the time when people use it, it actually slows things down. Using it, you're introducing network latency and serialization overhead. Sometimes that's worth it, especially if your database is falling over, but a lot of the time people use it and it just makes everything more complex and worse. If you need to share cached data across processes or nodes, sometimes you have to use it, but a lot of the stuff I work with is partitioned anyway. If your data is already partitioned, you know what works well a lot of the time? A boring, regular hashmap. Pretty much every language has some thread-safe hashmap in there, and a lot of them have pretty decent libraries to handle invalidation and expiration if you need those. In Java, for example, you have ConcurrentHashMap for simple stuff, and Guava Caches or Caffeine Caches for more advanced stuff. Even the slowest [1] local caching implementation will almost certainly be faster than anything that hits the network; in my own testing [2] Caffeine caches have sub-microsecond `put` times, and you don't pay any serialization or deserialization cost. I don't think you're likely to get much better than maybe sub-millisecond times with Redis, even in the same data center, not to mention if you're caching locally that's one less service that you have to babysit. Again, I don't hate Redis, there are absolutely cases where it's a good fit, I just think it's overused. [1] Realistic I mean, obviously any of use could artificially construct something that is slow as we want. [2] https://blog.tombert.com/posts/2025-03-06-microbenchmark-errata/ https://blog.tombert.com/posts/2025-03-06-microbenchmark-err... This is my own blog, feel free to not click it. Not trying to plug myself, just citing my data.

hajimuz 2y ago
In most cases It’s not about the speed, it’s about data sharing for containers or distributed systems. Filesystem or in-memory doesn’t work. I agree that in most cases a normal database is enough though.
- tombert 2y ago
  Yeah I mentioned that, if you need to share stuff between processes or differnet nodes, then maybe Redis might be a fit. But I've seen people use Redis as a glorified "global variable" for stuff like Kafka streaming. The data is already partitioned, it's not going to be used across multiple nodes, and now you've introduced another service to look at and made everything slower because of the network. A global hashmap (or cache library, like previously mentioned) would do the job faster, with less overhead, and the code would be simpler.

jayd16 2y ago

I agree that Redis can be an architectural smell. You can shoehorn it in where it doesn't belong. It's easy to work with and does it's job so well that for a certain type of dev it's easier to implement caching than understand enough SQL to add proper indices.

tengbretson 2y ago
Its funny, because once you go through the effort of determining which query fields you should compose to make your cache key, you're 95% of the way to the index you should have had in the first place.

bdcravens 2y ago

Another category is using Redis indirectly via dependencies. For example in Rails, Sidekiq is a common background job library. However, there are now Postgresql-backed options (like GoodJob and the baked-in Solid Queue, which supports other RDBMSes as well)

jherdman 2y ago
I just wanted to take a moment to highlight both GoodJob and Solid Queue. They're excellent choices, and any RoR folks reading these comments should give them a fair shake.

sgarland 2y ago

> 'Why can't we just store this data on the shards in PostgreSQL, next to the swipes?'. The data itself would be microscopic in comparison and the additional load would also be microscopic in comparison to what these servers were already doing. I'm assuming based on the rest of the article that the author and team knew what they were doing, but if you aren't familiar with Postgres' UPDATE strategy and HOT updates [0], you should familiarize yourself before attempting this, otherwise, you're going to generate a massive amount of WAL traffic and dead tuples at scale. [0]: https://www.postgresql.org/docs/current/storage-hot.html https://www.postgresql.org/docs/current/storage-hot.html

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

igortg 2y ago

I followed with this rationale in a small project and opted for PostgreSQL pub/sub instead Redis. But I went through so much trouble correctly handling PostgreSQL disconnections that I wonder if Redis wouldn't be the better choice.

phendrenad2 2y ago

I like to run a single redis server process on each instance that's serving the webapp, and also have a shared redis cluster for shared cached state (like user sessions). That way there's a fast local cache for people to use for caching API calls and other things where duplication is preferable to latency.

tengbretson 2y ago
If it's purely ephemeral and not replicated between instances what are you getting from redis that couldn't be done with SQLite or even a good old hashmap in a separate thread?
- phendrenad2 2y ago
  Redis is probably slightly faster than sqlite, and developers are used to using redis primitives and APIs.
- ricardobeat 2y ago
  You get standardized data structures and consistent operations on them for free.

kflgkans 2y ago

You might not need a cache. In my previous company (~7 years) all teams around me were introducing caches left and right and getting into a lot of complexity and bugs. I persevered and always pushed back adding caches to apps in my team. Instead focusing on improving the architecture and seeking other performance improvements. I can proudly say my teams have stayed cached-free for those 7 years.

xyzzy9563 2y ago

Redis is useful if you have multiple backend servers that need to synchronize state. If you don’t have this problem then improving your caching I.e. using Cloudflare is probably the best approach, along with local caching on a single server.

g9yuayon 2y ago

When I was in Uber back in 2015, my org was trying to convert zip-code-based geo partitioning with a hexagon-based scheme. Instead of partitioning a city into on average tens of zip codes, we may partition the city into potentially hundreds of thousands of hexagons and dynamically create areas. The first launch was in Phoenix, and the team who was responsible for the launch stayed up all night for days because they could barely scale our demand-pricing systems. And then the global launch of the feature was delayed first by days, then by weeks, and then by months. It turned out Uber engineers just loved Redis. Having a need to distribute your work? Throw that to Redis. I remember debating with some infra engineers why we couldn't throw in more redis/memcached nodes to scale our telemetry system, but I digressed. So, the price service we built was based on Redis. The service fanned out millions of requests per second to redis clusters to get information about individual hexagons of a given city, and then computed dynamic areas. We would need dozens of servers just to compute for a single city. I forgot the exact number, but let's say it was 40 servers per an average-sized city. Now multiply that by the 200+ cities we had. It was just prohibitively expensive, let alone that there couldn't other scalability bottlenecks for managing such scale. The solution was actually pretty simple. I took a look at the algorithms we used, and it was really just that we needed to compute multiple overlapping shapes. So, I wrote an algorithm that used work-stealing to compute the shapes in parallel per city on a single machine, and used Elasticsearch to retrieve hexagons by a number of attributes -- it was actually a perfect use case for a search engine because the retrieval requires boolean queries of multiple attributes. The rationale was pretty simple too: we needed to compute repetitively on the same set of data, so we should retrieve the data only once for multiple computations. The algorithm was of merely dozens of lines, and was implemented and deployed to production over the weekend by this amazing engineer Isaac, who happens to be the author of the library H3. As a result, we were able to compute dynamic areas for 40 cities, give or take, on a single machine, and the launch was unblocked.

muggermuch 2y ago
Cool anecdote, thanks for sharing!
pstuart 2y ago
The aforementioned hex mapping tool: https://h3geo.org/ https://h3geo.org/ For anyone doing geo queries it's a powerful tool.

lukaslalinsky 2y ago

For years I've tried to use Redis as a persistent data store. I've only been dissatisfied, having bad experiences with both sentinel and cluster. Most of my service outages were linked to Redis replication getting broken. Then I decided to give up and use it only as an empehemral cache. I have a large number of standalone Redis instances (actually, now they are Valkey), no storage, only memory, and have Envoy proxy on top of them for monitoring and sharding. And I'm really happy with it, storing hundreds of GBs of data there, if one goes down, only a small part of the data needs to be reloaded from the primary source, and with the Envoy proxy, applications see it as a single Redis server. I was considering just replacing it with memcached, but the redis data model is more rich, so I kept using it, just not expecting anything put there to be actually stored forever.

anonymousDan 2y ago
What were the problems you had when using it as a persistent data store if you don't mind me asking? Also, what kind of workloads did you have?
- lukaslalinsky 2y ago
  The problematic use case was using Redis as a counter for various app related metrics, which I want to make visible to the customer later on. App was writing to Redis, background job was reading these partially aggregated metrics from Redis and writing them to another database. So you could say it was a remote buffer. On way too many occasions, after short failure of the host where the current master instance was running, the replica was not propagated to master. That was using redis sentinel. I guess my main issue was always redis sentinel.

progbits 2y ago

Redis as ephemeral cache is ok, but nothing extra. Redis as transactional, distributed and/or durable storage is pretty poor. Their "active active" docs on conflict resolution for example don't fill me with confidence given there is no formalism, just vague examples. But this comes from people who not only not know how do distributed locks, they refuse to learn when issues are pointed out to them: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html https://martin.kleppmann.com/2016/02/08/how-to-do-distribute... Every time I find code that claims to do something transactional in Redis which is critical for correctness, not just latency optimization, I get worried.

ent101 2y ago

Shameless plug for my project: https://github.com/HeyPuter/kv.js https://github.com/HeyPuter/kv.js Don't kill me :')

0xblinq 2y ago
That's a great project! I'm definitely going to give it a try.

vvpan 2y ago

Really the most useful feature in Redis to me is TTL - so easy to cache things and not worry about them being too stale. The second most useful is that you can attach something like BullMQ to it and get cron/scheduling for a couple lines of code.

superq 2y ago

The issues that I have with Redis are not at all its API (which is elegant and brilliant) or even its serialized, single-core, single-threaded design, but its operational hazards. As a cache or ephemeral store like a throttling/rate limiting, lookup tables, or perhaps even sessions store, it's great; but it's impossible to rely on the persistence options (RDB, AOF) for production data stores. You usually only see this tendency with junior devs, though. It might be a case where "when all you have is a hammer, all you see are nails", or when someone discovers Redis (or during the MongoDB hype cycle ten years ago), which seems like it's in perfect alignment with their language datatypes, but perhaps this is mostly because junior devs don't have as many production-ready databases (from SQL like Postgresql, CockroachDB, Yugabyte to New/NoSQL like ScyllaDB, YDB, Aerospike) to fall back on. Redis shines as a cache for small data values (probably switch to memcache for larger values, which is simpler key-value but generally 3 to 10 times faster for that more narrow use case, although keep an eye on memory fragmentation and slab allocation) Just think carefully before storing long-term data in it. Maybe don't store your billing database in it :)

noisy_boy 2y ago
I have seen horrifying use of Redis where I inherited the maintainance of an application whose original developer implemented his own home grown design to manage relationships between different types of key value pairs, pretending like they were tables including cross-referencing logic; it took me a week to just add test cases with sufficient logging to reveal the "schema" and mutation logic. All this with the non-technical manager wondering why it took so long to make the change which directly dependended on understanding this. To top it all, the code was barely better than spaghetti with less than ten lines of comments across maybe 5k LOC. The irony was that this was not a latency sensitive application - it did data quality checks and could have been implemented in a much more cleaner and flexible way using, e.g., PostgreSQL.
- intelVISA 2y ago
  Non-technical 'manager'

0xbadcafebee 2y ago

Software development today is largely just people repeating what other people do without thinking. Which is how human culture works; we just copy what everyone else is doing, because it's easier, and that becomes "normal", whatever it is. In the software world in the mid 00's, the trend started to work around the latency, cost and complexity of expensive servers and difficult databases by relying on the speed of modern networks and RAM. This started with Memcached and moved on to other solutions like Redis. (this later evolved into NoSQL, when developers imagined that simply doing away with the complexity of databases would somehow magically remove their applications' need to do complex things... which of course it didn't, it's the same application, needing to do a complex thing, so it needs a complex solution. computers aren't magic. we have thankfully passed the hype cycle of NoSQL, and moved on to... the hype cycle for SQLite) But the tradeoff was always working around one limitation by adding another limitation. Specifically it was avoiding the cost of big databases and the expertise to manage them, and accepting the cost of dealing with more complex cache control. Fast forward to 2025 and databases are faster (but not a ton faster) and cheaper (but not a ton cheaper) and still have many of the same limitations (because dramatically reinventing the database would have been hard and boring, and no software developer wants to do hard and boring things, when they can do hard and fun things, or ignore the hard things with cheap hacks and pretend there is no consequence to that). So people today just throw a cache in between the database, because 1) databases are still kind of stupid and hard (very very useful, but still stupid and hard) and 2) the problems of cache complexity can be ignored for a while, and putting off something hard/annoying/boring until later is a human's favorite thing. No, you don't need Redis. Nobody needs Redis. It's a hack to avoid dealing with stateless applications using slow queries on an un-optimized database with no fast read replicas and connection limits. But that's normal now.

briandear 2y ago
Recently just left a MongoDB project. A total nightmare.
edoceo 2y ago
> hype cycle for SQLite Drop Redis, replace with in-memory SQLite. But for real, the :memory: feature is actually pretty awesome!

khaki54 2y ago

I was reading an article last week where the author did a bunch of testing and found Redis is only 3-5x faster than just using a multi model NoSQL. Basically it rarely makes sense to use Redis. https://adamfowler.org/2015/09/07/hybrid-nosql-marklogic-as-a-k-v-store/ https://adamfowler.org/2015/09/07/hybrid-nosql-marklogic-as-...

dcmatt 2y ago
"only 3-5x faster... rarely makes sense to use Redis" This is an illogical conclusion. 3-5x faster is substantial!
- lucb1e 2y ago
  I agree, although it also depends on the context. If a page needs to fetch lots of data and takes 3 full seconds to load, the middle case (4x improvement) makes it go from an amount of time where you glance at your messages to on par with the average javascript-laden website we're used to anyway (except in this case it's legitimate loading time). If, however, we're talking about most pages, that are mostly static and just fetch some article contents and author name from the DB in 2 milliseconds... yeah, it can be 10x faster and nobody cares

harrall 2y ago

Two things: I think people forget (or don’t know) that adding data storage system to your architecture also involves management, scaling, retention, and backup. It’s not free. And second, sometimes you do need to invest in storage to permit failover or to minimize latency but people do it for traffic when they really have little traffic. A server from 20 years ago could push a lot of traffic and systems have gotten only beefier.

antirez 2y ago

I believe that the issue is that the culture about Redis usage didn't evolve as much as its popularity. To use it memcached alike has many legitimate use cases but it's a very reductive way to use it. For instance sorted set ranking is something that totally changes the dynamics of what you can and can't do with traditional databases. Similarly large bitmaps that allow to retain very fast real time one bit information to do analytics otherwise very hard to do is another example. Basically, Redis helps a lot more as the company culture around it increases, more patterns are learned, and so forth. But in this regard a failure on the Redis (and my) side is that there isn't a patterns collection book: interviewing folks that handled important use cases (think at Twitter) to understand the wins and the exact usage details and data structure usages. Even just learning the writable cache pattern totally changes the dynamics of your Redis experience.

kshitij_libra 2y ago
Do you plan to write the book ? I’d like to read it
- mleonhard 2y ago
  Yes, please write it. I will buy and read it.
  - jordiburgos 2y ago
    I'll buy it, read it and take notes.

notjoemama 2y ago

> The first usage was to cache the result of an external API call to lookup geolocation of users, so that the service in question could process location requests quicker, and also run cheaper. BAAAHHH HAAHA HA HA HA HHHAA. Ok, look. Just because it's in Redis does not disqualify the clause in the geo service's license that you NOT permanently store the data. The author did not say that's what they were doing but a very similar thing came up in a previous work place for me and we chose to call the service for every lookup the way the service company expected their customers to do. I recall the suggestion in using a caching database as a workaround and it was not made by the engineering team. Sorry, I'm still chuckling at this one...

inib 2y ago
We use Redis to store geocoding data from Google Geocoding API with a TTL of 30 days, per the policy. It's $4/1000 requests. There'd no business without the cache.
dimgl 2y ago
I don't understand this comment. Without caching, you'd be at the mercy of the geolocation company. I've deployed a service that does exactly this.

rc_mob 2y ago

Yeah I just removed redis from my php app. It runs slightly slower but not enough that my user would notice. Redis was overkill

simonw 2y ago

If you work at a company where teams keep on turning to Redis for different features, there's a chance that it's an indication that the process for creating new database tables in your relational store has too much friction!

alberth 2y ago

> A single beefy Redis (many cores, lots of RAM type of machine) should be able to handle the load I thought Redis was single threaded running on a single core. Having multiple cores provides no benefit (and arguably could hurt since large multicore systems typically have a lower clock)

sriku 2y ago
Redis itself, yes. However there are Redis compatible tools that go beyond single core - KeyDB, DragonFly and ARDB overcome some limitations.

cyberax 2y ago

Redis (or Valkey) has a bunch of functionality other than caching. It has a very useful event streaming system and task queueing system. You can use Postgres instead of them, but then it can easily become a bottleneck.

alberth 2y ago

This post seems less about Redis, and more broadly about - why you should never introduce new technologies into your existing stack (only do so when your existing stack can’t keep up).

briandear 2y ago

Rails adding SolidCache and SolidQueue (or Good Job) was what made me not even think of Redis anymore in the rails world.

dimgl 2y ago

I'm really surprised that the pendulum has swung so far in the other direction that people are recommending not to use Redis. Sure, don't introduce a data store into your stack unless you need it. But if you had to introduce one, Redis still seems like one of the best to introduce? It has fantastic data structures (like sorted sets, hash maps), great performance, robust key expiry, low communication overhead, low runtime overhead... I mean, the list goes on.

esafak 2y ago
I'd like to draw attention to its probabilistic data structures, in particular: HyperLogLog, Bloom filter, Cuckoo filter, t-digest, Top-K, Count-min sketch https://redis.io/docs/latest/develop/data-types/probabilistic/ https://redis.io/docs/latest/develop/data-types/probabilisti...
- dimgl 2y ago
  This is wild, I had no idea it even had this.
ks2048 2y ago
Yeah, my first thought was, “I don’t need redis, but I want redis”.
hot_gril 2y ago
Or a simple KV cache, which is pretty often something you need. Sure you can use Postgres for this if you really don't want another dep, but Redis or Memcached is better suited for it and probably cheaper.

high_na_euv 2y ago

Case 1 Why not store this value on client side and sync it from server from time to time? Or it wasnt used by client?

deleted 2y ago

[deleted]

hackburg 2y ago

[dead]