5 ms·
I would love to have a Redis alternative where I don't have to think about whether or not the dataset fits into memory.
by akie 3y ago
I would love to have a Redis alternative where I don't have to think about whether or not the dataset fits into memory.
- deleted 3y ago[deleted]
- jetbalsa 3y agoI've used SSDB[0] in the past for some really stupid large datasets (20TB)_and it worked really well in production [0] https://github.com/ideawu/ssdb https://github.com/ideawu/ssdb
- PlutoIsAPlanet 3y agoIts also worth checking out kvrocks, which is a redis interface on top of rockdb that's part of the Apache project, and very well maintained.
- mathfailure 3y agoAnd which is not in-memory at all.
- PlutoIsAPlanet 3y agoIt can cache in-memory using RocksDBs caching mechanisms.
- welder 3y agoI switched from SSDB to Kvrocks recently, because SSDB is abandoned and the author missing for 3 years now. I used to recommend SSDB, but now there's better alternatives available: https://github.com/apache/kvrocks https://github.com/apache/kvrocks https://github.com/sabledb-io/sabledb https://github.com/sabledb-io/sabledb
- akie 3y agoThese are great recommendations, thanks!
- yuppiepuppie 3y agoCurious, What’s the use case?
- qwertox 3y agoMy same thought, because some don't require much memory.
- akie 3y agoThe use case is caching 20 million API responses that almost never change, each about 20kb of JSON, for a high traffic site. Yes, I can pay for a 400Gb RAM instance of Redis, but it's expensive. I can also cache it on disk, but then I need to think about cache expiration myself. Or I can use something appropriate like a document database, but then I need additional code & additional configuration because we otherwise don't need that piece of infrastructure in our stack. It would be a lot easier if I could just store it in Redis with the other (more reasonably sized) things that I need to cache.
- emmanueloga_ 3y agoCheck memcached "extstore" [1]. -- 1: https://github.com/memcached/memcached/wiki/Extstore https://github.com/memcached/memcached/wiki/Extstore
- Nican 3y agoThis looks like a good use case for ScyllaDB with Compression and TTL. It is pretty simple to setup a single-node instance. If you rather have something in-process and writes to disk, to avoid extra infrastructure, I would also recommend RocksDB with Compression and TTL.
- 0cf8612b2e1e 3y agoWould DiskCache work for you? Runs via SQLite memory or persisted file database. Thread safe, has various expiration controls, etc. https://grantjenks.com/docs/diskcache/tutorial.html https://grantjenks.com/docs/diskcache/tutorial.html
- GraemeMeyer 3y agoI think Garnet might be what you're looking for: https://www.microsoft.com/en-us/research/project/garnet/ https://www.microsoft.com/en-us/research/project/garnet/
- Nican 3y agoAs usual, there is a spectrum of data safety vs. performance. Redis is at the "very fast, but unsafe" side of the scale. ScyllaDB for me is in the middle of being high performance key-value store, but not really supporting transactions. FoundationDB is another one that I would consider.
- tyre 3y agoDepends on the kind of safety you’re looking for. Redis is entirely safe from concurrency issues because it’s single-threaded. It supports an append-only file for persistence to disk.
- alerighi 3y agoAt that point why using Redis entirely? You can use any DBMS you want, either relational or NoSQL. The advantage of Redis is that it is a memory cache, if you take out the memory from it, just use Postgres or whatever DBMS you are using (I say Postgres because it has all the features of Redis).
- dalyons 3y agoPostgres has nowhere near all the features of redis. Go and have a look at the redis command’s documentation. They’re not even really similar at all, once you get past basic GET/SET stuff.
- from-nibly 3y agoCan you name an explicit thing that postgres does not do that redis does?
- jitl 3y agoThis is silly, Postgres doesn’t speak the Redis wire protocol. You will need a large army of connection proxies to get a Postgres database to handle the number of connections a single Redis shrugs off with no sweat. Maybe you like this answer more: At the end of the day you can embed a bunch of Turing-complete programming languages in Postgres, and Postgres can store binary blobs, so Postgres can do literally anything. Can it do it performantly, and for low cost? Probably not. But if you put in enough time and money I’m sure you can re-implement Redis on Postgres using BLOB column alone. Here’s a simpler answer: cuckoo filter is available out of the box in Redis, 2 seconds of Googling I didn’t find one for Postgres: https://redis.io/docs/latest/develop/data-types/probabilistic/cuckoo-filter/ https://redis.io/docs/latest/develop/data-types/probabilisti...
- codetrotter 3y agoNot sure if this one could be used in order to do what you want but maybe? https://www.postgresql.org/docs/current/bloom.html https://www.postgresql.org/docs/current/bloom.html Have a look