4 ms·

I've been watching RethinkDB and it looks really cool, but... One need I have in nearly every project is record versioning. Think implementing a modern wiki, i

by porker 12y ago

I've been watching RethinkDB and it looks really cool, but... One need I have in nearly every project is record versioning. Think implementing a modern wiki, in a CRM (what did X alter? Which phone numbers have been assigned to this person?), versioning updates to data records etc. Realtime web with multiple users editing records simultaneously, it's even more important. Versioning is a pain to implement in every system I've used, and I'm looking for something to lower that pain. CouchDB et al have built-in versioning, but cause pain in other places. I've looked at the RethinkDB docs and found no mention - so how would RethinkDB users handle versioning, and are there any helpers on the way?

raspasov 12y ago
I am working on something like this, feel free to ping me @raspasov
UserRights 12y ago
yes, versioning please for all the databases, missing this since years! Please more talk about this!
- dkersten 12y ago
  CouchDB and Datomic are the only databases that I know about that have built-in versioning. Would love to hear about others that do too!
  - juliangregorian 12y ago
    MVCC is not versioning and you can not rely on CouchDB's MVCC functionality to provide versioning. Datomic on the other hand does claim to provide "rewindability".
    - dkersten 12y ago
      Ah, Ok, I stand corrected. I only used CouchDB very very briefly some years ago and was under the illusion that its document version numbering could be used to keep track of document changes. I didn't realise it was purely for MVCC purposes. Yes, Datomic advertises that it keeps track of a documents history.
nkohari 12y ago
The app I'm building uses incremented version counters as a way of protecting against simultaneous edits. I'm discarding the old version of each document, but it would be pretty trivial to save them using the "returnChanges" option of r.update() [1]. If you set returnChanges, the result will contain two properties, old_val and new_val. You could take the contents of old_val, give it a unique identifier, and then store it in another table with r.create(). [1] More about r.update(): http://rethinkdb.com/api/javascript/update/ http://rethinkdb.com/api/javascript/update/
juliangregorian 12y ago
MVCC is not versioning. MVCC is concurrency control. I would have to be convinced that the DB is an appropriate place to handle versioning of the kind you are speaking of. For instance, see this article: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-collection/ http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-g... Upshot is that RethinkDB started out append-only with rewindability etc. and then ran into real-world problems. Every system I've ever seen that did versioning does it at the software level and stores multiple records in the database. Rails gem "paper trail" is an example.
- porker 12y ago
  That's a fair point. I sat down and tried to sketch it on paper after posting my comment, and found there was too much domain knowledge involved to do versioning directly in the database. My comment can be rephrased that it's still painful to do this at the application level: for such a common task, the frameworks/libraries/software I work with don't have anything baked in to handle it. I've started trying to use Git as an additional data backend to hold the versioning, but changing from a JSON/Database storage structure to one Git will happily diff and reconcile is in itself not pretty.
  - juliangregorian 12y ago
    SQLite might be a tolerable solution for that, as long as you can stop the world between commits.
neumino 12y ago
You can add a field "last_updated", and update a document with the option {returnChanges: true}, and you'll get the new and old values. You can then save the old values in another table like "history" where the primary key is `_id` instead of `id`. In this case, you can do r.table("data").get(1) // get the current version of the document with id 1 r.table("history").getAll(1, {index: "id"}) // get all the previous versions of the document with id 1. And you can order them by `last_updated` if you want too.

by porker 12y ago

raspasov 12y ago

I am working on something like this, feel free to ping me @raspasov

UserRights 12y ago

yes, versioning please for all the databases, missing this since years! Please more talk about this!

dkersten 12y ago
CouchDB and Datomic are the only databases that I know about that have built-in versioning. Would love to hear about others that do too!
- juliangregorian 12y ago
  MVCC is not versioning and you can not rely on CouchDB's MVCC functionality to provide versioning. Datomic on the other hand does claim to provide "rewindability".
  - dkersten 12y ago
    Ah, Ok, I stand corrected. I only used CouchDB very very briefly some years ago and was under the illusion that its document version numbering could be used to keep track of document changes. I didn't realise it was purely for MVCC purposes. Yes, Datomic advertises that it keeps track of a documents history.

nkohari 12y ago

The app I'm building uses incremented version counters as a way of protecting against simultaneous edits. I'm discarding the old version of each document, but it would be pretty trivial to save them using the "returnChanges" option of r.update() [1]. If you set returnChanges, the result will contain two properties, old_val and new_val. You could take the contents of old_val, give it a unique identifier, and then store it in another table with r.create(). [1] More about r.update(): http://rethinkdb.com/api/javascript/update/ http://rethinkdb.com/api/javascript/update/

juliangregorian 12y ago

MVCC is not versioning. MVCC is concurrency control. I would have to be convinced that the DB is an appropriate place to handle versioning of the kind you are speaking of. For instance, see this article: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-collection/ http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-g... Upshot is that RethinkDB started out append-only with rewindability etc. and then ran into real-world problems. Every system I've ever seen that did versioning does it at the software level and stores multiple records in the database. Rails gem "paper trail" is an example.

porker 12y ago
That's a fair point. I sat down and tried to sketch it on paper after posting my comment, and found there was too much domain knowledge involved to do versioning directly in the database. My comment can be rephrased that it's still painful to do this at the application level: for such a common task, the frameworks/libraries/software I work with don't have anything baked in to handle it. I've started trying to use Git as an additional data backend to hold the versioning, but changing from a JSON/Database storage structure to one Git will happily diff and reconcile is in itself not pretty.
- juliangregorian 12y ago
  SQLite might be a tolerable solution for that, as long as you can stop the world between commits.

neumino 12y ago

You can add a field "last_updated", and update a document with the option {returnChanges: true}, and you'll get the new and old values. You can then save the old values in another table like "history" where the primary key is `_id` instead of `id`. In this case, you can do r.table("data").get(1) // get the current version of the document with id 1 r.table("history").getAll(1, {index: "id"}) // get all the previous versions of the document with id 1. And you can order them by `last_updated` if you want too.