2 ms·

Redis is as reliable as the storage you persist it to. If you're running Redis right, it's very reliable. Not S3 reliable, though. But if you need S3 reliable,

Redis is as reliable as the storage you persist it to. If you're running Redis right, it's very reliable. Not S3 reliable, though. But if you need S3 reliable, you would turn to something else. I expect that most folks looking at this are doing it because it means: 1. Effectively unbounded storage 2. It's fast 3. It's pretty darn cheap 4. You can scale it horizontally in a way that's challenging to scale other filesystems 5. All the components are pretty easy to set up. Many folks are probably already running S3 and Redis.

danpalmer 9mo ago
Redis isn't durable unless you drastically reduce the performance. Filesystems are pretty much by definition durable.
- bastawhiz 9mo ago
  Enabling the WAL doesn't make Redis slow. It's slower than the default, but it's still exceptionally fast. > Filesystems are pretty much by definition durable. Where do you think Redis persists its data to
  - danpalmer 9mo ago
    Redis using the WAL for durable data can only be as fast as you can fsync on every write, just like every other database. The time to fsync is going to dominate the write path for Redis, and you'll sacrifice almost all of Redis's speed to achieve it. If you're running Redis in always fsync mode it I'd suggest there are better storage systems to use. > Where do you think Redis persists its data to This is sort of the point, Redis isn't for persistence. I mean it can, you can use the WAL, it's handy for not having to recompute when you relaunch Redis etc, but Redis is fundamentally an in-memory system and should be treated as such. It is designed for use-cases that necessitate in-memory performance, and that don't require durability.
doctorpangloss 9mo ago
> 4. You can scale it horizontally in a way that's challenging to scale other filesystems Easy to scale on RDS, along with everything else. But there’s no Kubernetes operator. Is there a better measure of “easy” or “challenging?” IMO no. Perhaps I am spoiled by CNPG.
staticassertion 9mo ago
> Redis is as reliable as the storage you persist it to. For a single node, if you tank performance by changing the configuration, sure. Otherwise, no, not really. I don't get why you'd want a file system that isn't durable, but to each their own.
- bastawhiz 9mo ago
  With the wal redis runs with perfectly reasonable performance. Of course you're not going to have the performance of an in-memory only DB if you're flushing to disk on every write. There's no vacuuming, there's no need for indexing. You can see the time complexity of most operations. Key-value operations are mostly O(1). You'll never get that kind of performance with other databases because they intentionally don't give you that granularity. The metadata of the filesystem isn't the performance bottleneck in most cases.
  - staticassertion 9mo ago
    > With the wal redis runs with perfectly reasonable performance. Of course you're not going to have the performance of an in-memory only DB if you're flushing to disk on every write. But that's not what gets benchmarked against. > You'll never get that kind of performance with other databases because they intentionally don't give you that granularity. Okay but you also won't get horizontal scaling while maintaining consistency in Redis. > The metadata of the filesystem isn't the performance bottleneck in most cases. Okay, but they do show that other metadata stores are 2-4x slower than Redis, so it would be great to see whole-system benchmarks that use those.

danpalmer 9mo ago

Redis isn't durable unless you drastically reduce the performance. Filesystems are pretty much by definition durable.

bastawhiz 9mo ago
Enabling the WAL doesn't make Redis slow. It's slower than the default, but it's still exceptionally fast. > Filesystems are pretty much by definition durable. Where do you think Redis persists its data to
- danpalmer 9mo ago
  Redis using the WAL for durable data can only be as fast as you can fsync on every write, just like every other database. The time to fsync is going to dominate the write path for Redis, and you'll sacrifice almost all of Redis's speed to achieve it. If you're running Redis in always fsync mode it I'd suggest there are better storage systems to use. > Where do you think Redis persists its data to This is sort of the point, Redis isn't for persistence. I mean it can, you can use the WAL, it's handy for not having to recompute when you relaunch Redis etc, but Redis is fundamentally an in-memory system and should be treated as such. It is designed for use-cases that necessitate in-memory performance, and that don't require durability.

doctorpangloss 9mo ago

> 4. You can scale it horizontally in a way that's challenging to scale other filesystems Easy to scale on RDS, along with everything else. But there’s no Kubernetes operator. Is there a better measure of “easy” or “challenging?” IMO no. Perhaps I am spoiled by CNPG.

staticassertion 9mo ago

> Redis is as reliable as the storage you persist it to. For a single node, if you tank performance by changing the configuration, sure. Otherwise, no, not really. I don't get why you'd want a file system that isn't durable, but to each their own.

bastawhiz 9mo ago
With the wal redis runs with perfectly reasonable performance. Of course you're not going to have the performance of an in-memory only DB if you're flushing to disk on every write. There's no vacuuming, there's no need for indexing. You can see the time complexity of most operations. Key-value operations are mostly O(1). You'll never get that kind of performance with other databases because they intentionally don't give you that granularity. The metadata of the filesystem isn't the performance bottleneck in most cases.
- staticassertion 9mo ago
  > With the wal redis runs with perfectly reasonable performance. Of course you're not going to have the performance of an in-memory only DB if you're flushing to disk on every write. But that's not what gets benchmarked against. > You'll never get that kind of performance with other databases because they intentionally don't give you that granularity. Okay but you also won't get horizontal scaling while maintaining consistency in Redis. > The metadata of the filesystem isn't the performance bottleneck in most cases. Okay, but they do show that other metadata stores are 2-4x slower than Redis, so it would be great to see whole-system benchmarks that use those.