4 ms·

Document Stores: Please Give Me A Standard API

9 points by lecha 17y ago

lecha 17y ago
The author focuses on document stores, but the same argument can be made to key/value stores. It is time for NoSQL community to come up with a standard key/value API. Is anyone working on it? As one commented said "JDBC came out ~25 years after SQL was created.". By today's standards 2.5 years should be about right time to expect a standard :)
- cperciva 17y ago
  It is time for NoSQL community to come up with a standard key/value API. Is anyone working on it? I think if people want a standard API, it's going to be the memcached API.
  - _pius 17y ago
    For key-value stores, probably. For document stores with richer functionality, probably not.
    - cperciva 17y ago
      Well, yes. But the question was specifically about key-value APIs.
- antirez 17y ago
  Can't talk for the other NoSQL players, but for Redis this is more or less impossible since the key-value business is something like 10% of the current API. How to cover the other 90%? Maybe in the future Redis may be able to listen in another port to talk the memcached protocol, but this is not an high priority feature for now. What I think people should do is to abstract the interaction with the DB in their code. So that it's just a matter of writing adapters to support new KV stores, and this adapters can take advantage of peculiarities in specific stores. For instance if using Redis one can use Redis Sets to take the list of friends. When using something different will serialize data as JSON, and so forth.
  - gthank 17y ago
    Since a lot of these different data stores have fairly different semantics, I'm not sure how much value there is in trying to create a common set of abstractions in your code.
    - antirez 17y ago
      There is a lot of value if the DB API is abstract enough, like: (id) addBookmark(title,url,taglist) (bookmark) getBookmark(id) (array) searchBookmarksByTag(taglist) And so forth. You can switch from SQL to a NOSQL solution and so forth without to touch the app but just a single file with all the DB API. If it's at very low level like Db.get(), DB.set(), ... is still useful if the application just using a strict common subset of features (get/set/exists/expire/incr).
iamaleksey 17y ago
A standard might lead to slower innovation at this point.
moe 17y ago
Yes. SQL should be that standard API. Most document- and KV-stores only support a subset of what can be expressed with SQL. In fact, I have yet to see a feature that can not be expressed in SQL. Thus, the advantages of using SQL are obvious: * SQL is well understood and mature * People already know SQL * SQL parsers and clients are dime a dozen * SQL lends itself reasonably well to interactive use by humans. Much better than typing raw javascript into a console (MongoDB) or having to write actual code for every little data-manipulation task (most others). * Existing SQL-infrastructure can be leveraged. For example your favorite ORM could easily grow an adapter for a "NoSQL"-Db when it's just a slightly different SQL dialect instead of a completely distinct API.
- lecha 17y ago
  Good points, but SQL is 'only' a language, not a protocol. Servers still need to agree how to expose an HTTP API that could take SQL. Ideally such API would be RESTful.
  - moe 17y ago
    I don't see that as a big problem. The queries will have to be parsed and processed inside the Database anyways. Thus the servers could simply accept SQL strings over HTTP - or any other transport protocol.
- evgen 17y ago
  > SQL should be that standard API. Unlikely. How will SQL deal with low-level semantics that do not provide transactions in the way SQL-users understand them but requires the user to understand the concepts of the CAP principle? How well does SQL deal with conflicting data being returned from retrieval operation? If SQL is being used as nothing more than a small set of verbs (a la memcache) then why bother? I certainly hope that document-store developers do not take the lazy route and prematurely converge on a standard like SQL. Fortunately it seems that few of them are particularly interested in this route and the more likely path seems to be providing multiple interfaces with RESTful HTTP, memcache, and a programmable/functional interface using JavaScript and/or the language the DB is coded in (Erlang and Java primarily.) Other possible candidates here are things like LINQ, Hive's QL, and Pig.

9 points by lecha 17y ago

lecha 17y ago

The author focuses on document stores, but the same argument can be made to key/value stores. It is time for NoSQL community to come up with a standard key/value API. Is anyone working on it? As one commented said "JDBC came out ~25 years after SQL was created.". By today's standards 2.5 years should be about right time to expect a standard :)

cperciva 17y ago
It is time for NoSQL community to come up with a standard key/value API. Is anyone working on it? I think if people want a standard API, it's going to be the memcached API.
- _pius 17y ago
  For key-value stores, probably. For document stores with richer functionality, probably not.
  - cperciva 17y ago
    Well, yes. But the question was specifically about key-value APIs.
antirez 17y ago
Can't talk for the other NoSQL players, but for Redis this is more or less impossible since the key-value business is something like 10% of the current API. How to cover the other 90%? Maybe in the future Redis may be able to listen in another port to talk the memcached protocol, but this is not an high priority feature for now. What I think people should do is to abstract the interaction with the DB in their code. So that it's just a matter of writing adapters to support new KV stores, and this adapters can take advantage of peculiarities in specific stores. For instance if using Redis one can use Redis Sets to take the list of friends. When using something different will serialize data as JSON, and so forth.
- gthank 17y ago
  Since a lot of these different data stores have fairly different semantics, I'm not sure how much value there is in trying to create a common set of abstractions in your code.
  - antirez 17y ago
    There is a lot of value if the DB API is abstract enough, like: (id) addBookmark(title,url,taglist) (bookmark) getBookmark(id) (array) searchBookmarksByTag(taglist) And so forth. You can switch from SQL to a NOSQL solution and so forth without to touch the app but just a single file with all the DB API. If it's at very low level like Db.get(), DB.set(), ... is still useful if the application just using a strict common subset of features (get/set/exists/expire/incr).

iamaleksey 17y ago

A standard might lead to slower innovation at this point.

moe 17y ago

Yes. SQL should be that standard API. Most document- and KV-stores only support a subset of what can be expressed with SQL. In fact, I have yet to see a feature that can not be expressed in SQL. Thus, the advantages of using SQL are obvious: * SQL is well understood and mature * People already know SQL * SQL parsers and clients are dime a dozen * SQL lends itself reasonably well to interactive use by humans. Much better than typing raw javascript into a console (MongoDB) or having to write actual code for every little data-manipulation task (most others). * Existing SQL-infrastructure can be leveraged. For example your favorite ORM could easily grow an adapter for a "NoSQL"-Db when it's just a slightly different SQL dialect instead of a completely distinct API.

lecha 17y ago
Good points, but SQL is 'only' a language, not a protocol. Servers still need to agree how to expose an HTTP API that could take SQL. Ideally such API would be RESTful.
- moe 17y ago
  I don't see that as a big problem. The queries will have to be parsed and processed inside the Database anyways. Thus the servers could simply accept SQL strings over HTTP - or any other transport protocol.
evgen 17y ago
> SQL should be that standard API. Unlikely. How will SQL deal with low-level semantics that do not provide transactions in the way SQL-users understand them but requires the user to understand the concepts of the CAP principle? How well does SQL deal with conflicting data being returned from retrieval operation? If SQL is being used as nothing more than a small set of verbs (a la memcache) then why bother? I certainly hope that document-store developers do not take the lazy route and prematurely converge on a standard like SQL. Fortunately it seems that few of them are particularly interested in this route and the more likely path seems to be providing multiple interfaces with RESTful HTTP, memcache, and a programmable/functional interface using JavaScript and/or the language the DB is coded in (Erlang and Java primarily.) Other possible candidates here are things like LINQ, Hive's QL, and Pig.