5 ms·
I'm one of the lead programmers of Keyspace, which is a key-value store. The main strength of Keyspace is the consistent replication, but right now I'll tell yo
by Maro 17y ago
I'm one of the lead programmers of Keyspace, which is a key-value store. The main strength of Keyspace is the consistent replication, but right now I'll tell you what my experiences are building a web app (real-time search) using Keyspace. In other words, what happened when I drank my own kool-aid.
1. Schemaless. This is a half-truth. In reality there is always some kind of schema, otherwise you'd get chaos. What works for us in Keyspace, where everything is a string, is to use "/" as a separator convention. So we have keys like:
"/user/id:001" => <data>
More complicated example:
"/user/id:001/view/keyword:keyspace/"
2. Listing. Since listing happens on a prefix basis in Keyspace (give me everything that starts with "/user/id:001"), if you just want to get a list of some properties of the user, you have to create the list explicitly, otherwise you'll get all the keys under the user. Eg.
"/user/id:001/views/list/keyword:scalien" => null
"/user/id:001/views/list/keyword:keyspace" => null
3. If you need to list by property X, you need to create a subtree for it, as above. This sucks if you figure this out at a later point, because you have to programatically parse your existing data and create it, besides adding it to the online code for new items. That's why RDBMSs are so good.
4. Backups. No good backup strategy at this point, hot or cold.
..
So why should anyone put up with this? You have to realize there's lots of hype right now, in reality KV stores should only be used by a small fraction of DB users. If an RDBMS is Java/C++, then a KV store is (can be) Assembly. You have much more control over what's going on. (You can of course get the same control by having a giant key-value table in your RDBMS.) You can put a cache layer in your stack more easily.
So you should think of a good KV store as something that you use under certain hi-performance cases, and even then, the KV store should offer some plus over having a giant key-value table in My/Postresql. Usually, that plus is some kind of distribution or replication that usually works. In case of Keyspace it's replication.
The other big plus is, as I learned, is that programming in the KV model is much much more natural then having embedded SQL statements in your code. A typical segment in my (Python) code is:
users = conn.listkeys("/user/id:%s/views/" % user_id)
for (...):
This is much nicer then having SQL statements, and since you're using a KV store, it can't get more complicated than this. As I said, I assume you're doing hi-performance stuff, in which case some crappy ORM layer is out of the question anyway.
- msluyter 17y agoHave you made, or you aware of, any comparisons between a KV store such as this and the same thing implemented as an IOT in Oracle?
- Maro 17y agoNo and no. Here's a link about IOT for others: http://www.oracle.com/technology/products/oracle9i/datasheets/iots/iot_ds.html http://www.oracle.com/technology/products/oracle9i/datasheet... Personally I think there's a fairly large cultural disconnect between shops running open-source stacks and shops running Oracle, so such a comparison would not influence user choices much. But it would be very interesting for an implementor like me: if Oracle is faster, which is likely at this point, it would set a performance point to beat. As a final fun fact, Keyspace is currently running on top of BerkeleyDB, which is an Oracle property =) But not for much longer, I'm very unsatisfied with BDB, it doesn't live up to its image of being an industrial strength btree store (it sucks).
- msluyter 17y agoPersonally I think there's a fairly large cultural disconnect between shops running open-source stacks and shops running Oracle... Quite possibly. Our company does use a lot of open source in the Java & Perl layers, but we're pretty tightly locked in to Oracle on the backend. I think it may be true that the sheer vastness of Oracle tempts you into believing you can solve every problem the Oracle way, which is why IOTs came to mind. I'd love to experiment with other options, myself, but its difficult once everything you do is predicated on it.
- gcv 17y agoWhat do you dislike about Berkeley DB? I've been using it for a little while for a pet project, and it seems decent. I'd like it to support range-based join cursors, but that's not a major complaint. Granted, I haven't yet had to deal with database recovery, so I don't know how it stacks up there. Also --- what do you plan to replace it with? I know that AllegroCache started out based on BDB, and ended up not using it. The Franz guys said performance wasn't up to par, and I always wished for more details.