5 ms·
I'm a big fan of both Redis and SQLite, so I decided to combine the two. SQLite is specifically designed for many small queries[1], and it's probably as close a
by nalgeon 3y ago
I'm a big fan of both Redis and SQLite, so I decided to combine the two. SQLite is specifically designed for many small queries[1], and it's probably as close as relational engines can get to Redis, so I think it might be a good fit.
[1]: https://sqlite.org/np1queryprob.html https://sqlite.org/np1queryprob.html
- sesm 3y agoWhat are the project goals? I assume it's a drop-in replacement for Redis that is supposed to be better in certain cases? If yes, then what cases do you have in mind?
- nalgeon 3y agoThe goal is to have a convenient API to work with common data structures, with an SQL backend and all the benefits it provides. Such as: — Small memory footprint even for large datasets. — ACID transactions. — SQL interface for introspection and reporting.
- sesm 3y agoSo the goal is to have a Redis-like API but not actually be an in-memory data store and reduce memory consumption this way? For example, for a project/service that started with Redis, but then priorities shifted and small memory footprint became more important than performance? Did I get it right?
- grobbyy 3y agoI can tell you my use case. I have a lot of systems which I'd like to use on a single computer AND at scale. That's sometimes even just for development work. A lot of these use a common API to a more complex distributed store, as well as to something simple like files on disk, in memory, or SQLite. I'm most cases, it's one user at a time, so performance doesn't matter, but simplicity does. It can also be for the project which has a 1 percent chance of going viral. Etc. But I find relatively few cases between truly small scale and large scale.
- sesm 3y agoLocal dev drop-in replacement for Redis seems like a really good usecase, thanks!
- b33j0r 3y agoI love this, it’s the solution that makes sense for 90% of the times I have used redis with python. I’ve made several versions of this, and to be honest, it ended up being so straightforward that I assumed it was a trivial solution. This is pretty well-planned. This is 100% the way to go. Heh. I took a detour into making my idea of “streams” also solve event sourcing in native python; dumb idea, if interesting. Mission creep probably killed my effort! Nice work
- yawaramin 3y agoAny plans to support Redis' memory-based eviction policy eg `volatile-lru`? To me a great benefit of Redis is its guarantee of stable memory usage.
- justinclift 3y agoThat's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ? https://python-rq.org This RQ stuff has been a pain with a recent project because only Python seems to use it, so once an RQ job has been submitted only Python based things can do anything with it. :( If Redka works as a backend replacement, we could potentially have non-Python things check the SQLite database instead.
- Spivak 3y agoHighly recommend Faktory (by the folks who brought you Sidekiq) as a language agnostic job server.
- justinclift 3y agoThanks, looks potentially interesting: https://github.com/contribsys/faktory https://github.com/contribsys/faktory And it's written in Go. :)
- nalgeon 3y agoIt works with redis-py (which python-rq uses), but I doubt it will be any good in this case. python-rq seems to use Lua scripting in Redis, which is not planned for 1.0. I'd rather not add it at all, but we'll see.
- justinclift 3y agoNo worries. :)