4 ms·

Are there any major libraries for OT? I've been looking into this recently for a project at work, and OT would be completely sufficient for our use case, and d

by MrJohz 7mo ago

Are there any major libraries for OT? I've been looking into this recently for a project at work, and OT would be completely sufficient for our use case, and does look simpler overall, but from what I could tell, we'd need to write a lot of stuff ourselves. The only vaguely active-looking project in JS at least seems to be DocNode (https://www.docukit.dev/docnode https://www.docukit.dev/docnode), and that looks very cool but also very early days.

samlinnfer 7mo ago
https://github.com/josephg/sharejs https://github.com/josephg/sharejs https://github.com/FirebaseExtended/firepad https://github.com/FirebaseExtended/firepad https://github.com/Operational-Transformation/ot.js https://github.com/Operational-Transformation/ot.js https://github.com/ottypes/docs https://github.com/ottypes/docs Really nice demo: https://operational-transformation.github.io https://operational-transformation.github.io
antics 7mo ago
Author here. I think it depends what you're doing! OT is a true distributed systems algorithm and to my knowledge there are no projects that implement true, distributed OT with strong support for modern rich text editor SDKs like ProseMirror. ShareJS, for example, is abandoned, and predates most modern editors. If you are using a centralized server and ProseMirror, there are several OT and pseudo-OT implementations. Most popularly, there is prosemirror-collab[4], which is basically "OT without the stuff you don't need with an authoritative source for documents." Practically speaking that means "OT without T", but because it does not transform the ops to be order-independent, it has an extra step on conflict where the user has to rebase changes and re-submit. This is can cause minor edit starvation of less-connected clients. prosemirror-collab-commit[5] fixes this by performing the rebasing on the server... so it's still "OT without the T", but also with an authoritative conflict resolution pseudo-T at the end. I personally recommend prosemirror-collab-commit, it's what we use, and it's extremely fast and predictable. If you just want something pedogocically helpful, the blessed upstream collaborative editing solution for CodeMirror is OT. See author's blog post[1], the @codemirror/collab package[2], and the live demo[3]. In general this implementation is quite good and worth reading if you are interested in this kind of thing. ShareJS and OTTypes are both very readable and very good, although we found them very challenging to adopt in a real-world ProseMirror-based editor. [1]: https://marijnhaverbeke.nl/blog/collaborative-editing-cm.html https://marijnhaverbeke.nl/blog/collaborative-editing-cm.htm... [2]: https://codemirror.net/docs/ref/#collab https://codemirror.net/docs/ref/#collab [3]: https://codemirror.net/examples/collab/ https://codemirror.net/examples/collab/ [4]: https://github.com/ProseMirror/prosemirror-collab https://github.com/ProseMirror/prosemirror-collab [5]: https://github.com/stepwisehq/prosemirror-collab-commit https://github.com/stepwisehq/prosemirror-collab-commit
- MrJohz 7mo ago
  In our case, we're not using a text editor, but instead building a spreadsheet, so a lot of these collab-built-into-an-editor are, like you say, pedagogically useful but less helpful as direct building blocks that we can just pull in and use. But the advice is very useful, thank you!
  - nhatcher 7mo ago
    Interesting! I am building a spreadsheet and the next few months will be building the collaborative side of it. I think many of the things that work for text don't necessarily translate for spreadsheets.
    - josephg 7mo ago
      We made a spreadsheet on top of OT several years ago. Most OT related documentation doesn't talk about how to do this. But it worked pretty well for us.
- ghProTip 7mo ago
  Cheers for plugging prosemirror-collab-commit! Nice to see it's getting used more.
- gritzko 7mo ago
  When I was starting my research into collaborative editing as a PhD student 20+ years ago, rebase-and-resubmit was well known. It was used in one Microsoft team collab product (I forgot the name). It is 100% legit algo except intermittently-connected clients may face challenges (screw them then). Unless you have to support some complicated scenarios, it will work. I believe Google Docs initially used something of the sort (diff-match-patch based). It annoyed users with alerts "lets rebase your changes", esp on bad WiFi. So they borrowed proper OT from Google Wave and lived happily since (not really). One way to think about it: how many users will your product have and how strange your data races / corner cases can get. At Google's scale, 0.1% users complaining is a huge shit storm. For others, that is one crazy guy in the channel, no biggie. It all depends. TLDR: people invented OT/CRDT for a reason.
GermanJablo 7mo ago
Author of DocNode here. Yes, it’s still early days. But it’s a very robust library that I don’t expect will go through many breaking changes. It has been developed privately for over 2 years and has 100% test coverage. Additionally, each test uses a wrapper to validate things like operation reversibility, consistency across different replicas, etc. DocSync, which is the sync engine mainly designed with DocNode in mind, I would say is a bit less mature. I’d love it if you could take a look and see if there’s anything that doesn’t convince you. I’ll be happy to answer any questions.
- MrJohz 7mo ago
  I've looked through the site, and right now it's probably the thing I'd try out first, but my main concerns are the missing documentation, particular the more cookbook-y kinds of documentation — how you might achieve such-and-such effect, etc. For example, the sync example is very terse, although I can understand why you'd like to encourage people to use the more robust, paid-for solution! Also just general advice on how to use DocNode effectively from your experience would be useful, things like schema design or notes about how each operation works and when to prefer one kind of operation or structure over another. All that said, I feel like the documentation has improved since the last time I looked, and I suspect a lot of the finer details come with community and experience.
  - GermanJablo 7mo ago
    Thanks! I've recently made some improvements to the documentation. I agree the synchronization section could be improved more. I'll keep your feedback in mind. If you'd like to try the library, feel free to ask me anything on Discord and I'll help you.
vesselapi 7mo ago
[dead]

by MrJohz 7mo ago

samlinnfer 7mo ago

https://github.com/josephg/sharejs https://github.com/josephg/sharejs https://github.com/FirebaseExtended/firepad https://github.com/FirebaseExtended/firepad https://github.com/Operational-Transformation/ot.js https://github.com/Operational-Transformation/ot.js https://github.com/ottypes/docs https://github.com/ottypes/docs Really nice demo: https://operational-transformation.github.io https://operational-transformation.github.io

antics 7mo ago

Author here. I think it depends what you're doing! OT is a true distributed systems algorithm and to my knowledge there are no projects that implement true, distributed OT with strong support for modern rich text editor SDKs like ProseMirror. ShareJS, for example, is abandoned, and predates most modern editors. If you are using a centralized server and ProseMirror, there are several OT and pseudo-OT implementations. Most popularly, there is prosemirror-collab[4], which is basically "OT without the stuff you don't need with an authoritative source for documents." Practically speaking that means "OT without T", but because it does not transform the ops to be order-independent, it has an extra step on conflict where the user has to rebase changes and re-submit. This is can cause minor edit starvation of less-connected clients. prosemirror-collab-commit[5] fixes this by performing the rebasing on the server... so it's still "OT without the T", but also with an authoritative conflict resolution pseudo-T at the end. I personally recommend prosemirror-collab-commit, it's what we use, and it's extremely fast and predictable. If you just want something pedogocically helpful, the blessed upstream collaborative editing solution for CodeMirror is OT. See author's blog post[1], the @codemirror/collab package[2], and the live demo[3]. In general this implementation is quite good and worth reading if you are interested in this kind of thing. ShareJS and OTTypes are both very readable and very good, although we found them very challenging to adopt in a real-world ProseMirror-based editor. [1]: https://marijnhaverbeke.nl/blog/collaborative-editing-cm.html https://marijnhaverbeke.nl/blog/collaborative-editing-cm.htm... [2]: https://codemirror.net/docs/ref/#collab https://codemirror.net/docs/ref/#collab [3]: https://codemirror.net/examples/collab/ https://codemirror.net/examples/collab/ [4]: https://github.com/ProseMirror/prosemirror-collab https://github.com/ProseMirror/prosemirror-collab [5]: https://github.com/stepwisehq/prosemirror-collab-commit https://github.com/stepwisehq/prosemirror-collab-commit

MrJohz 7mo ago
In our case, we're not using a text editor, but instead building a spreadsheet, so a lot of these collab-built-into-an-editor are, like you say, pedagogically useful but less helpful as direct building blocks that we can just pull in and use. But the advice is very useful, thank you!
- nhatcher 7mo ago
  Interesting! I am building a spreadsheet and the next few months will be building the collaborative side of it. I think many of the things that work for text don't necessarily translate for spreadsheets.
  - josephg 7mo ago
    We made a spreadsheet on top of OT several years ago. Most OT related documentation doesn't talk about how to do this. But it worked pretty well for us.
ghProTip 7mo ago
Cheers for plugging prosemirror-collab-commit! Nice to see it's getting used more.
gritzko 7mo ago
When I was starting my research into collaborative editing as a PhD student 20+ years ago, rebase-and-resubmit was well known. It was used in one Microsoft team collab product (I forgot the name). It is 100% legit algo except intermittently-connected clients may face challenges (screw them then). Unless you have to support some complicated scenarios, it will work. I believe Google Docs initially used something of the sort (diff-match-patch based). It annoyed users with alerts "lets rebase your changes", esp on bad WiFi. So they borrowed proper OT from Google Wave and lived happily since (not really). One way to think about it: how many users will your product have and how strange your data races / corner cases can get. At Google's scale, 0.1% users complaining is a huge shit storm. For others, that is one crazy guy in the channel, no biggie. It all depends. TLDR: people invented OT/CRDT for a reason.

GermanJablo 7mo ago

Author of DocNode here. Yes, it’s still early days. But it’s a very robust library that I don’t expect will go through many breaking changes. It has been developed privately for over 2 years and has 100% test coverage. Additionally, each test uses a wrapper to validate things like operation reversibility, consistency across different replicas, etc. DocSync, which is the sync engine mainly designed with DocNode in mind, I would say is a bit less mature. I’d love it if you could take a look and see if there’s anything that doesn’t convince you. I’ll be happy to answer any questions.

MrJohz 7mo ago
I've looked through the site, and right now it's probably the thing I'd try out first, but my main concerns are the missing documentation, particular the more cookbook-y kinds of documentation — how you might achieve such-and-such effect, etc. For example, the sync example is very terse, although I can understand why you'd like to encourage people to use the more robust, paid-for solution! Also just general advice on how to use DocNode effectively from your experience would be useful, things like schema design or notes about how each operation works and when to prefer one kind of operation or structure over another. All that said, I feel like the documentation has improved since the last time I looked, and I suspect a lot of the finer details come with community and experience.
- GermanJablo 7mo ago
  Thanks! I've recently made some improvements to the documentation. I agree the synchronization section could be improved more. I'll keep your feedback in mind. If you'd like to try the library, feel free to ask me anything on Discord and I'll help you.

vesselapi 7mo ago

[dead]