5 ms·

> We were quite seriously scared of repeating the infamous history of Netscape which was a well-known example of failing to successfully release a newer version

> We were quite seriously scared of repeating the infamous history of Netscape which was a well-known example of failing to successfully release a newer version of a popular software after deciding to rewrite it from scratch. Fortunately it did not happen in our case. Big kudos for pulling it off! > Two main candidates are Operational Transformation (OT) and Conflict-Free Replicated Data Type (CRDT). We chose OT and perhaps one day we will write down our thoughts on the ongoing OT vs. CRDT battle. Please do, I would love to hear your thoughts on OT vs CRDT!

sroussey 8y ago
Ditto. Would love to hear more about the OT vs CRDT decision!
- toomim 8y ago
  I ditto your ditto. (Is this now a tritto?)
  - wll 8y ago
    «Ditto is a library for using CRDTs.» [0] [0] https://github.com/alex-shapiro/ditto https://github.com/alex-shapiro/ditto
- codebeaker 8y ago
  You both may enjoy http://archagon.net/blog/2018/03/24/data-laced-with-history/ http://archagon.net/blog/2018/03/24/data-laced-with-history/ if you haven't seen it already, it's an absolute gem, and touches on exactly this.
  - codetrotter 8y ago
    Thank you for the suggestion, that was a very enjoyable read indeed :) Thanks to the author as well, for distilling so much information into the very readable document that this was. One thing I am wondering about is in the conclusion of the document, the author says: > You’d be hard-pressed to use CRDTs in data-heavy scenarios such as screen sharing or video editing. With regards to video editing, could it not be rather doable after all? Consider a non-linear video editing system (NLVE) [1] like Adobe Premiere Pro [2]. While the size of the video data source material you are editing might range into probably several hundreds of gigabytes for a 4K feature length film, the project file itself remains very small, because the video data is kept separate from the timeline data. Wikipedia explains: > Non-destructive editing is a form of audio, video, or image editing in which the original content is not modified in the course of editing; instead the edits are specified and modified by specialized software. A pointer-based playlist, effectively an edit decision list (EDL), for video or a directed acyclic graph for still images is used to keep track of edits. Are these not in fact the sorts of data structures that would lend themselves the most naturally to being implemented as CRDTs? :D As for the video source material that is used in the project, those files can be distributed by users outside of the editor software using for example BitTorrent, or they could be automatically distributed by the editor software via for example public IPFS or a private Dropbox folder, or if the users that are working together are on a shared network then the source files could be located on a shared NAS. [1]: https://en.wikipedia.org/wiki/Non-linear_editing_system https://en.wikipedia.org/wiki/Non-linear_editing_system [2]: https://en.wikipedia.org/wiki/Adobe_Premiere_Pro https://en.wikipedia.org/wiki/Adobe_Premiere_Pro
Reinmar 8y ago
Oh, that's really tempting :) Believe us! There are people in our team who would love to dive deeper into CRDT. We confirm that OT definitely has its problems. It's elegant at first but gets really dirty the deeper you dive into this... hole. Especially the undo feature adds to it a tone of complexity. So, some of my colleagues see CRDT as a potential saviour. There was even a paper on CRDT for tree structures that we might use (can't find a link now). However, we'd rather go with a linear model for CRDT. That'd make it a bit easier on the CRDT level but it would require some really smart decisions about: types of changes, the structure of the linear model and how it is transformed to the tree model (that we need after all). We know, though, that without an actual implementation, it's just a theoretical discussion. Therefore, we started thinking whether we could add CRDT "below" our current tree model. That might allow validating the concept in a real-life. Can't wait to do that :) BTW, in the meantime, we recommend this paper: https://arxiv.org/abs/1810.02137 https://arxiv.org/abs/1810.02137
- scofalik 8y ago
  Here's the link to the paper discussing tree-based CRDT: https://arxiv.org/abs/1201.1784 https://arxiv.org/abs/1201.1784. I personally find those papers very interesting!
skosch 8y ago
For anyone interested in CRDT, I can recommend the deep-dive writeup to Raph Levien's Xi editor: https://xi-editor.github.io/xi-editor/docs.html https://xi-editor.github.io/xi-editor/docs.html
k1w1 8y ago
As a gentle introduction to OTs, one of my colleagues did a Rails Conf 2018 presentation on implementing collaborative editing. His blog is here https://blog.aha.io/text-editor/ https://blog.aha.io/text-editor/ including a simple implementation in Ruby.

sroussey 8y ago

Ditto. Would love to hear more about the OT vs CRDT decision!

toomim 8y ago
I ditto your ditto. (Is this now a tritto?)
- wll 8y ago
  «Ditto is a library for using CRDTs.» [0] [0] https://github.com/alex-shapiro/ditto https://github.com/alex-shapiro/ditto
codebeaker 8y ago
You both may enjoy http://archagon.net/blog/2018/03/24/data-laced-with-history/ http://archagon.net/blog/2018/03/24/data-laced-with-history/ if you haven't seen it already, it's an absolute gem, and touches on exactly this.
- codetrotter 8y ago
  Thank you for the suggestion, that was a very enjoyable read indeed :) Thanks to the author as well, for distilling so much information into the very readable document that this was. One thing I am wondering about is in the conclusion of the document, the author says: > You’d be hard-pressed to use CRDTs in data-heavy scenarios such as screen sharing or video editing. With regards to video editing, could it not be rather doable after all? Consider a non-linear video editing system (NLVE) [1] like Adobe Premiere Pro [2]. While the size of the video data source material you are editing might range into probably several hundreds of gigabytes for a 4K feature length film, the project file itself remains very small, because the video data is kept separate from the timeline data. Wikipedia explains: > Non-destructive editing is a form of audio, video, or image editing in which the original content is not modified in the course of editing; instead the edits are specified and modified by specialized software. A pointer-based playlist, effectively an edit decision list (EDL), for video or a directed acyclic graph for still images is used to keep track of edits. Are these not in fact the sorts of data structures that would lend themselves the most naturally to being implemented as CRDTs? :D As for the video source material that is used in the project, those files can be distributed by users outside of the editor software using for example BitTorrent, or they could be automatically distributed by the editor software via for example public IPFS or a private Dropbox folder, or if the users that are working together are on a shared network then the source files could be located on a shared NAS. [1]: https://en.wikipedia.org/wiki/Non-linear_editing_system https://en.wikipedia.org/wiki/Non-linear_editing_system [2]: https://en.wikipedia.org/wiki/Adobe_Premiere_Pro https://en.wikipedia.org/wiki/Adobe_Premiere_Pro

Reinmar 8y ago

Oh, that's really tempting :) Believe us! There are people in our team who would love to dive deeper into CRDT. We confirm that OT definitely has its problems. It's elegant at first but gets really dirty the deeper you dive into this... hole. Especially the undo feature adds to it a tone of complexity. So, some of my colleagues see CRDT as a potential saviour. There was even a paper on CRDT for tree structures that we might use (can't find a link now). However, we'd rather go with a linear model for CRDT. That'd make it a bit easier on the CRDT level but it would require some really smart decisions about: types of changes, the structure of the linear model and how it is transformed to the tree model (that we need after all). We know, though, that without an actual implementation, it's just a theoretical discussion. Therefore, we started thinking whether we could add CRDT "below" our current tree model. That might allow validating the concept in a real-life. Can't wait to do that :) BTW, in the meantime, we recommend this paper: https://arxiv.org/abs/1810.02137 https://arxiv.org/abs/1810.02137

scofalik 8y ago
Here's the link to the paper discussing tree-based CRDT: https://arxiv.org/abs/1201.1784 https://arxiv.org/abs/1201.1784. I personally find those papers very interesting!

skosch 8y ago

For anyone interested in CRDT, I can recommend the deep-dive writeup to Raph Levien's Xi editor: https://xi-editor.github.io/xi-editor/docs.html https://xi-editor.github.io/xi-editor/docs.html

k1w1 8y ago

As a gentle introduction to OTs, one of my colleagues did a Rails Conf 2018 presentation on implementing collaborative editing. His blog is here https://blog.aha.io/text-editor/ https://blog.aha.io/text-editor/ including a simple implementation in Ruby.