5 ms·

I see that Remutable uses 'patches'. How does it compare with ShareJS? Does it allow collaborative editing (ie multiple people changing the same data structure

I see that Remutable uses 'patches'. How does it compare with ShareJS? Does it allow collaborative editing (ie multiple people changing the same data structure concurrently, is patch application commutative)? The idea behind ShareJS (or OT for that matter) is awesome. You never have to think about what to do if two people edit the same document concurrently. It just works. You get undo for free. And you can write the code that it saves automatically in the background. Never again do users have to press a 'save' button. A while ago I wrote a library which builds on top of the same ideas as ShareJS (ie. OT) but with slightly different focus. You define object types and their properties. This allows the library to automatically parse JSON into JavaScript class instances. It also tracks changes to these objects in a very similar way as ShareJS: each change has a 'path' (where something was changed) and then the actual chang record (currently set and splice). I use Avers in an SPA which is an editor of fairly complex data structures (90 different object types, nested ~5 levels deep). For rendering I use React. The integration is really simple: Attach a change listener to the root and re-render when anything changes. https://github.com/wereHamster/avers https://github.com/wereHamster/avers https://caurea.org/2015/02/08/the-future-of-avers.html https://caurea.org/2015/02/08/the-future-of-avers.html

elierotenberg 12y ago
I didn't now about ShareJS, but I'll definitely check it out. At its core, the patch abstraction is commutative (and more importantly, reversible), so I should be able to implement the 'rebase'-like operation. My feeling is that it would be more efficient to perform this at a higher level though, by replaying Actions in the right order in every client to get eventual consistency.
- wereHamster 12y ago
  ShareJS is just one implementation of OT. If you want to check something out, then start with OT (http://en.wikipedia.org/wiki/Operational_transformation http://en.wikipedia.org/wiki/Operational_transformation).
  - elierotenberg 12y ago
    Very interesting pointer, thanks. Remutable.Patch objects represent a collection of map edits, ie. a set of { key, value } pairs indicating that 'key' should be replaced by 'value' (and value can be void 0 to indicate deletion). They also embed a version and a hash for faster matching, but they can already be reverted/combined (for undo/redo stack or batching). I will read more about OTs!
    - aidos 12y ago
      OTs are the same idea along with a sort of rebase step for replaying changes that come in to get the head to the same point (regardless of the order in which the operations were received). It seems like a great model, but as I mentioned above, there's surprisingly little code around for you to draw on. I seem to recall reading a conversation from the ShareJS guys (though maybe it was someone else) about how hairy the edge cases can get when you're trying to do it for general data-structures (which was disappointing, because conceptually it seems simple).
- dugmartin 12y ago
  You should also checkout ot.js: https://github.com/Operational-Transformation/ot.js/ https://github.com/Operational-Transformation/ot.js/ They have a nice visualization of OT here: http://operational-transformation.github.io/visualization.html http://operational-transformation.github.io/visualization.ht...
aidos 12y ago
Interesting. I'm surprised at how little work seems to have been done around OT (operational transformation, for those who haven't heard of it before). I have an application where I hold a lot of synced data in the client and the server but I've struggled to find good libraries to help manage that. Generally the work seemed to be around text, rather than arbitrary objects. Sharejs was the best I could find, but my BE system is python (and the object support in ShareJS was a bit sketchy when I looked at it). Unrelated but looking at your project I saw you were using Sets and Maps. I didn't even know JS had those yet (and they're something I really miss coming from python). Will be switching immediately.
- elierotenberg 12y ago
  Native (mutable) Maps and Weakmaps are coming up with ES6, and are already (mostly) polyfilled by things like core.js. Immutable provides (immutable) implementations of Map, OrderedMap, Set, etc. See http://facebook.github.io/immutable-js/ http://facebook.github.io/immutable-js/ :)
  - aidos 12y ago
    I'm going to have a deeper dig through your code today too. These ideas are definitely along the lines of what I've had formulating in my mind recently so I'd like to see how you've gone about implementing it. Since it seems like you know a bit about this - do you have any experience with dealing with immutable structures in larger scale applications? I used a react for a sub project to my core app and one of the issues I ran into straight away was that you can't really keep any state within components because of the way they're destroyed and created (something as simple as a file tree with an open/closed flag on folders falls down). So you need to have all of your state in stores. What's the best way of managing that? I've been envisaging a world in which my application is one single large immutable data-structure containing everything about the current state of the application. The actions and then carried out (maybe via a backend over a websocket as you've described) to make changes to the tree. React then looks after the rendering.
    - calgaryeng 12y ago
unconed 12y ago
Indeed, Share.js is awesome. I've worked extensively with it, and recently hooked up Share.js to immutable React using a modified React-Cursor. It takes local mutations and converts them into JSON-OT. Remote OTs are merged back in, the server is always ground truth. You get local reactive UI with immutable state. On the server-side, you can validate the ops as they come in using e.g. the entire infrastructure of passport in node. OAuth2-validated Share.js with strict per-property write access, no problem. I also extended the couchDB driver to merge in server-side changes, thus allowing daemons to make live changes to the DB without having to care about any of the OT stuff. If share.js changes were made in the meantime, it will save a merged snapshot. Basically, Share.js can be your insurance policy against pesky humans messing up your data structures, in a world of atomic data-driven robots and asynchronous feeds.

elierotenberg 12y ago

I didn't now about ShareJS, but I'll definitely check it out. At its core, the patch abstraction is commutative (and more importantly, reversible), so I should be able to implement the 'rebase'-like operation. My feeling is that it would be more efficient to perform this at a higher level though, by replaying Actions in the right order in every client to get eventual consistency.

wereHamster 12y ago
ShareJS is just one implementation of OT. If you want to check something out, then start with OT (http://en.wikipedia.org/wiki/Operational_transformation http://en.wikipedia.org/wiki/Operational_transformation).
- elierotenberg 12y ago
  Very interesting pointer, thanks. Remutable.Patch objects represent a collection of map edits, ie. a set of { key, value } pairs indicating that 'key' should be replaced by 'value' (and value can be void 0 to indicate deletion). They also embed a version and a hash for faster matching, but they can already be reverted/combined (for undo/redo stack or batching). I will read more about OTs!
  - aidos 12y ago
    OTs are the same idea along with a sort of rebase step for replaying changes that come in to get the head to the same point (regardless of the order in which the operations were received). It seems like a great model, but as I mentioned above, there's surprisingly little code around for you to draw on. I seem to recall reading a conversation from the ShareJS guys (though maybe it was someone else) about how hairy the edge cases can get when you're trying to do it for general data-structures (which was disappointing, because conceptually it seems simple).
dugmartin 12y ago
You should also checkout ot.js: https://github.com/Operational-Transformation/ot.js/ https://github.com/Operational-Transformation/ot.js/ They have a nice visualization of OT here: http://operational-transformation.github.io/visualization.html http://operational-transformation.github.io/visualization.ht...

aidos 12y ago

Interesting. I'm surprised at how little work seems to have been done around OT (operational transformation, for those who haven't heard of it before). I have an application where I hold a lot of synced data in the client and the server but I've struggled to find good libraries to help manage that. Generally the work seemed to be around text, rather than arbitrary objects. Sharejs was the best I could find, but my BE system is python (and the object support in ShareJS was a bit sketchy when I looked at it). Unrelated but looking at your project I saw you were using Sets and Maps. I didn't even know JS had those yet (and they're something I really miss coming from python). Will be switching immediately.

elierotenberg 12y ago
Native (mutable) Maps and Weakmaps are coming up with ES6, and are already (mostly) polyfilled by things like core.js. Immutable provides (immutable) implementations of Map, OrderedMap, Set, etc. See http://facebook.github.io/immutable-js/ http://facebook.github.io/immutable-js/ :)
- aidos 12y ago
  I'm going to have a deeper dig through your code today too. These ideas are definitely along the lines of what I've had formulating in my mind recently so I'd like to see how you've gone about implementing it. Since it seems like you know a bit about this - do you have any experience with dealing with immutable structures in larger scale applications? I used a react for a sub project to my core app and one of the issues I ran into straight away was that you can't really keep any state within components because of the way they're destroyed and created (something as simple as a file tree with an open/closed flag on folders falls down). So you need to have all of your state in stores. What's the best way of managing that? I've been envisaging a world in which my application is one single large immutable data-structure containing everything about the current state of the application. The actions and then carried out (maybe via a backend over a websocket as you've described) to make changes to the tree. React then looks after the rendering.
  - calgaryeng 12y ago

unconed 12y ago

Indeed, Share.js is awesome. I've worked extensively with it, and recently hooked up Share.js to immutable React using a modified React-Cursor. It takes local mutations and converts them into JSON-OT. Remote OTs are merged back in, the server is always ground truth. You get local reactive UI with immutable state. On the server-side, you can validate the ops as they come in using e.g. the entire infrastructure of passport in node. OAuth2-validated Share.js with strict per-property write access, no problem. I also extended the couchDB driver to merge in server-side changes, thus allowing daemons to make live changes to the DB without having to care about any of the OT stuff. If share.js changes were made in the meantime, it will save a merged snapshot. Basically, Share.js can be your insurance policy against pesky humans messing up your data structures, in a world of atomic data-driven robots and asynchronous feeds.