3 ms·

Framework for what? You'll probably need to give us more details. For starters, will it be a mostly read-only or write-heavy database? What relative priorities

Framework for what? You'll probably need to give us more details. For starters, will it be a mostly read-only or write-heavy database? What relative priorities are you placing on speed, data consistency, ease of ad-hoc queries, etc.? Based on the few details you've given, I'd consider SQLite (http://sqlite.org/ http://sqlite.org/). It's relational, but very lightweight and runs in-process. It's also open-source, quite mature, simple to use, and interfaces very easily with C.

russell 18y ago
SQLite is suitable only for single user cases. "redis, memcachedb & tokyo cabinet" imply his needs are way beyond the capabilities of SQLite.
- silentbicycle 18y ago
  Not necessarily. If he's doing a lot of concurrent writing, that's true, but that's unspecified. Either way, I would say "only up to a couple dozen simultaneous users", which is very different from "only a single user". He's also asking specifically for something with low latency and (implicitly) a simple implementation. For all we know, he's using the above because he doesn't like (or understand?) SQL/relational theory, or because of vague stuff he keeps hearing about MySQL not scaling. I'm not assuming he's ignorant, but we don't know why he started with memcachedb, et al, and those reasons are worth ruling out first.
  - liuliu 18y ago
    the reason why not using sql is that the thing I am doing is more about using a "library" (like what tokyo cabinet c api provides). SQLite may be good for this purpose but still, I guess there will be a simpler implementation specially for attacking this kind of problem. MySQL/PgSQL is not suitable because of the authorization mechanism.
    - silentbicycle 18y ago
      SQLite is a C library, and is called in-process. It's probably the best fitting relational database for your problem, as I understand it. You've painted a pretty incomplete picture, though. The constraints on the problem (how much simultaneous r/w access, size of the data set, etc.) will have the most impact on what to choose, but it doesn't seem like you've fleshed that out anywhere else in the thread.
      - liuliu 18y ago
        in my scenario, it is a read-heavy database with occasionally writes. As average about 3 tags for 500,000 images, it should supports 1.5 million in-memory records on one computer. for performance, I expect ~100ms to retrieve 100,000 records.

russell 18y ago

SQLite is suitable only for single user cases. "redis, memcachedb & tokyo cabinet" imply his needs are way beyond the capabilities of SQLite.

silentbicycle 18y ago
Not necessarily. If he's doing a lot of concurrent writing, that's true, but that's unspecified. Either way, I would say "only up to a couple dozen simultaneous users", which is very different from "only a single user". He's also asking specifically for something with low latency and (implicitly) a simple implementation. For all we know, he's using the above because he doesn't like (or understand?) SQL/relational theory, or because of vague stuff he keeps hearing about MySQL not scaling. I'm not assuming he's ignorant, but we don't know why he started with memcachedb, et al, and those reasons are worth ruling out first.
- liuliu 18y ago
  the reason why not using sql is that the thing I am doing is more about using a "library" (like what tokyo cabinet c api provides). SQLite may be good for this purpose but still, I guess there will be a simpler implementation specially for attacking this kind of problem. MySQL/PgSQL is not suitable because of the authorization mechanism.
  - silentbicycle 18y ago
    SQLite is a C library, and is called in-process. It's probably the best fitting relational database for your problem, as I understand it. You've painted a pretty incomplete picture, though. The constraints on the problem (how much simultaneous r/w access, size of the data set, etc.) will have the most impact on what to choose, but it doesn't seem like you've fleshed that out anywhere else in the thread.
    - liuliu 18y ago
      in my scenario, it is a read-heavy database with occasionally writes. As average about 3 tags for 500,000 images, it should supports 1.5 million in-memory records on one computer. for performance, I expect ~100ms to retrieve 100,000 records.