4 ms·

Serious question: Why do people feel it's necessary to use a redis cluster? I understand in early 2000s we were using spinning disks and it was the only way. W

Serious question: Why do people feel it's necessary to use a redis cluster? I understand in early 2000s we were using spinning disks and it was the only way. Well, we don't use spinning disks any more, do we? A modern server can easily have terabytes of RAM and petabytes of NVMe, so what's stopping people from just using postgres? A cluster of radishes is an anti-pattern.

amtamt 4y ago
For caching somewhat larger objects based on ETag?
lofaszvanitt 4y ago
People know it, that's all.
cplli 4y ago
For caching the query results you get from your database. Also it's easier to spin up Redis and replicate it closer to your user than doing that with your main database. From my experience anyway.
- mike_hearn 4y ago
  I think the idea is that if your db can hold the working set in RAM and you're using a good db + prepared queries, you can just let it absorb the full workload because the act of fetching the data from the db is nearly as cheap as fetching it from redis.
- killerstorm 4y ago
  > For caching the query results you get from your database. This only makes sense if queries are computationally intensive. If you're fetching a single row by index you aren't winning much (or anything).
  - dpkirchner 4y ago
    Of course? I'm not really sure what the original question actually is if you know that users benefit from caching the results of computationally intensive queries.
    - killerstorm 4y ago
      OpenAI uses redis to store pieces of text. Fetching pieces of text is not computationally intensive.
      - mannyv 4y ago
        Most likely they have them in an rdbms, so it's more like joining a forum thread together. Not expensive, but why not prebuild and store it instead?
  - acuozzo 4y ago
    > This only makes sense if queries are computationally intensive. Or if the link to your DB is higher latency than you're comfortable with.
eldenring 4y ago
Yes! I have been spending the last couple months pulling out completely unnecessary redis caching from some of our internal web servers. The only loss here is network latency which negligible when you're colocated in AWS. Postgres's caches end up pulling a lot more weight too when you're not only hitting the db on a cache miss from the web server.
- deleted 4y ago
  [deleted]
manv1 4y ago
1. Redis can handle a lot more connections, more quickly, than a database can. 2. It's still faster than a database, especially a database that's busy. #2 is an interesting point. When you benchmark, the normal process is to just set up a database then run a shitload of queries against it. I don't think a lot of people put actual production load on the database then run the same set of queries against it...usually because you don't have a production load in the prototyping phase. However, load does make a difference. It made more of a difference in the HDD era, but it still makes a difference today. I mean, redis is a cache, and you do need to ensure that stuff works if your purge redis (ie: be sure the rebuild process works), etc, etc. But just because it's old doesn't mean it's bad. OS/390 and AS/400 boxes are still out there doing their jobs.
- nijave 4y ago
  A pretty small Redis server can handle 10k clients and saturate a 1Gbps NIC. You'd need a pretty heavy duty Postgres database and definitely need a connection pooler to come anywhere close.
  - anarazel 4y ago
    I agree that redis can handle some query volumes and client counts that postgres can't. But FWIW I can easily saturate a 10GBit ethernet link with primary key-lookup read-only queries, without the results being ridiculously wide or anything. Because it didn't need any setup, I just used: SELECT * FROM pg_class WHERE oid = 'pg_class'::regclass; I don't immediately have access to a faster network, connecting via tcp to localhost, and using some moderate pipelining (common in the redis world afaik), I get up to 19GB/s on my workstation.
    - e12e 4y ago
      > SELECT * FROM pg_class WHERE oid = 'pg_class'::regclass; This selects every column (*) from every table (ObjectID is of type regclass)?
      - 4y ago
xp84 4y ago
I'm confused on why the need to complicate something as seemingly-straightforward as a KV store into a series of queues that can get all mixed up. I asked ChatGPT to explain it though, and it sounds like the justification for its existence is that it doesn't "block the event loop" while a request is "waiting for a response from Redis." Last time I checked, Redis doesn't take that long to provide a response. And if your Redis servers actually are that overloaded that you're seeing latency in your requests, it seems like simple key-based sharding would allow horizontally scaling your Redis cluster. Disclaimer: I am probably less smart than most people who work at OpenAI so I'm sure I'm missing some details. Also this is apparently a Python thing and I don't know it beyond surface familiarity.
- zmj 4y ago
  I'm not familiar with the Python client specifically, but Redis clients generally multiplex concurrent requests onto a single connection per Redis server. That necessitates some queueing.
- oxymoron 4y ago
  Redis latency is around 1ms including network round trip for most operations. In a single threaded context, waiting on that would limit you to around 1000 operations per second. Redis clients improve throughput by doing pipelining, so a bunch of calls are batched up to minimize network roundtrips. This becomes more complicated in the context of redis-cluster, because calls targeting different keys are dispatched to different cache nodes and will complete in an unpredictable order, and additional client side logic is needed to accumulate the responses and dispatch them back to the appropiate caller.
adrr 4y ago
My redis clusters are 10x more cost effective than my postgresdb in handling load.
aadvark69 4y ago
Better concurrency (10k vs ~200 max connections compared to postgres). ~20x faster than Postgres at Key-value read/write operations. (mostly) single threaded, so atomicity is achieved without the synchronicity overhead found in RDBMS. Thus, it's much cheaper to run at massive scale like OpenAI's for certain workloads, including KV caching also: - robust, flexible data structures and atomic APIs to manipulate them are available out-of-the box - large and supportive community + tooling
YetAnotherNick 4y ago
Do you want to ask why we use caching instead of main db in RAM? Or why we use redis instead of postgres for caching?

amtamt 4y ago

For caching somewhat larger objects based on ETag?

lofaszvanitt 4y ago

People know it, that's all.

cplli 4y ago

For caching the query results you get from your database. Also it's easier to spin up Redis and replicate it closer to your user than doing that with your main database. From my experience anyway.

mike_hearn 4y ago
I think the idea is that if your db can hold the working set in RAM and you're using a good db + prepared queries, you can just let it absorb the full workload because the act of fetching the data from the db is nearly as cheap as fetching it from redis.
killerstorm 4y ago
> For caching the query results you get from your database. This only makes sense if queries are computationally intensive. If you're fetching a single row by index you aren't winning much (or anything).
- dpkirchner 4y ago
  Of course? I'm not really sure what the original question actually is if you know that users benefit from caching the results of computationally intensive queries.
  - killerstorm 4y ago
    OpenAI uses redis to store pieces of text. Fetching pieces of text is not computationally intensive.
    - mannyv 4y ago
      Most likely they have them in an rdbms, so it's more like joining a forum thread together. Not expensive, but why not prebuild and store it instead?
- acuozzo 4y ago
  > This only makes sense if queries are computationally intensive. Or if the link to your DB is higher latency than you're comfortable with.

eldenring 4y ago

Yes! I have been spending the last couple months pulling out completely unnecessary redis caching from some of our internal web servers. The only loss here is network latency which negligible when you're colocated in AWS. Postgres's caches end up pulling a lot more weight too when you're not only hitting the db on a cache miss from the web server.

deleted 4y ago
[deleted]

manv1 4y ago

1. Redis can handle a lot more connections, more quickly, than a database can. 2. It's still faster than a database, especially a database that's busy. #2 is an interesting point. When you benchmark, the normal process is to just set up a database then run a shitload of queries against it. I don't think a lot of people put actual production load on the database then run the same set of queries against it...usually because you don't have a production load in the prototyping phase. However, load does make a difference. It made more of a difference in the HDD era, but it still makes a difference today. I mean, redis is a cache, and you do need to ensure that stuff works if your purge redis (ie: be sure the rebuild process works), etc, etc. But just because it's old doesn't mean it's bad. OS/390 and AS/400 boxes are still out there doing their jobs.

nijave 4y ago
A pretty small Redis server can handle 10k clients and saturate a 1Gbps NIC. You'd need a pretty heavy duty Postgres database and definitely need a connection pooler to come anywhere close.
- anarazel 4y ago
  I agree that redis can handle some query volumes and client counts that postgres can't. But FWIW I can easily saturate a 10GBit ethernet link with primary key-lookup read-only queries, without the results being ridiculously wide or anything. Because it didn't need any setup, I just used: SELECT * FROM pg_class WHERE oid = 'pg_class'::regclass; I don't immediately have access to a faster network, connecting via tcp to localhost, and using some moderate pipelining (common in the redis world afaik), I get up to 19GB/s on my workstation.
  - e12e 4y ago
    > SELECT * FROM pg_class WHERE oid = 'pg_class'::regclass; This selects every column (*) from every table (ObjectID is of type regclass)?
    - 4y ago

xp84 4y ago

I'm confused on why the need to complicate something as seemingly-straightforward as a KV store into a series of queues that can get all mixed up. I asked ChatGPT to explain it though, and it sounds like the justification for its existence is that it doesn't "block the event loop" while a request is "waiting for a response from Redis." Last time I checked, Redis doesn't take that long to provide a response. And if your Redis servers actually are that overloaded that you're seeing latency in your requests, it seems like simple key-based sharding would allow horizontally scaling your Redis cluster. Disclaimer: I am probably less smart than most people who work at OpenAI so I'm sure I'm missing some details. Also this is apparently a Python thing and I don't know it beyond surface familiarity.

zmj 4y ago
I'm not familiar with the Python client specifically, but Redis clients generally multiplex concurrent requests onto a single connection per Redis server. That necessitates some queueing.
oxymoron 4y ago
Redis latency is around 1ms including network round trip for most operations. In a single threaded context, waiting on that would limit you to around 1000 operations per second. Redis clients improve throughput by doing pipelining, so a bunch of calls are batched up to minimize network roundtrips. This becomes more complicated in the context of redis-cluster, because calls targeting different keys are dispatched to different cache nodes and will complete in an unpredictable order, and additional client side logic is needed to accumulate the responses and dispatch them back to the appropiate caller.

adrr 4y ago

My redis clusters are 10x more cost effective than my postgresdb in handling load.

aadvark69 4y ago

Better concurrency (10k vs ~200 max connections compared to postgres). ~20x faster than Postgres at Key-value read/write operations. (mostly) single threaded, so atomicity is achieved without the synchronicity overhead found in RDBMS. Thus, it's much cheaper to run at massive scale like OpenAI's for certain workloads, including KV caching also: - robust, flexible data structures and atomic APIs to manipulate them are available out-of-the box - large and supportive community + tooling

YetAnotherNick 4y ago

Do you want to ask why we use caching instead of main db in RAM? Or why we use redis instead of postgres for caching?