6 ms·

This is where differential synchronization shines (basically git). While OT/CRDT are really good for syncing small edits, they suck at being eventually consiste

This is where differential synchronization shines (basically git). While OT/CRDT are really good for syncing small edits, they suck at being eventually consistent (with intentions preserved) on very large edits. We make a powerful collaborative word processor for the browser (Zoho Writer https://writer.zoho.com https://writer.zoho.com) with complete offline support. This is one problem we'd like to solve in the long term. One way to solve this is to fallback to differential sync when syncing huge offline edits - and present the changes as tracked-changes (underlined and striked-out) to the user. This way the user has the power to resolve the conflicts, instead of the app messing it up by itself. I'd like to hear if there are better ways to sync large offline edits, with the intentions preserved.

scofalik 7y ago
> One way to solve this is to fallback to differential sync when syncing huge offline edits This is another question -- does automatic syncing make sense for huge differences in users' contents? The answer might be no. OTOH, isn't this a rare scenario? How perfect it has to be?How much of a compromise can the users accept? These are interesting questions which shows that creating this kind of software isn't easy -- both from technical and UX point of view.
- jgtrosh 7y ago
  Well it's quite common for people using “normal” email-exchanged MS office revisions. I have no clue if there's specific tooling for these cases. Of course you could brush them off and say they wouldn't be a problem if they just used online editors.
- qpiox 7y ago
  First of all when discussing about offline edits we should make effort to draw attention that "offline" could mean different thing to different people. Offline editor from the point of view of the app's software developer might mean that editor can sometimes survive few minutes of lost connectivity, after which it will reconnect and possibly sync the team work. Offline from the perspective of a team of scientists writing a joint paper means that some of them will take their work truly offline, to a secluded mountain hut, for an extended period (week or month) and will come back with a complete rewrite of the text to the point of being unrecognizable to the other authors. And then the coauthors will disagree with most of it, and will want to revert some of the pieces and keep some other pieces. And scientists are not the only people needing joint collaborative environment. And not just for short papers. Few years ago, I was working in a larger team with a group of people doing revisions of a book, and there were so many revisions and revisions of revisions of revisions, and zillions of comments, that we had to split the book not in chapters, since Word got stuck even with 20-page chapters. We had to split the chapters in sub-chapters, of 5-10 pages in separate files, so that the processor would not get stuck with the myriad of comments and revisions. These are all separate scenarios in collaborative writing, and there could be many other scenarios, so any solution should first explain for what type of scenario is it really targeting. Automated sync of collaborative work is not always the best thing to do.
  - bluGill 7y ago
    Then there is the science fiction version where the editors are both online, but several light years apart with the resulting lag in updates.
bachmeier 7y ago
> sync large offline edits, with the intentions preserved How would that be possible if none of the users can see what the others are doing? If you have three users that start with version 1 and they all make their own version 2, even the users don't know what the "correct" version 2 should look like. Or to put it differently, I don't even know what is meant by "intentions" in this scenario.
baddox 7y ago
I work with a tool that provides diffing and merging for rich text (effectively a limited subset of HTML). It doesn’t have or need real-time collaborative functionality at all. I have read all the papers that I can find and understand, but I’m still not happy with our merging algorithm for rich text. The issue is less about which fancy algorithm/data structure to use and more about even defining in human terms what would be expected in certain merge conflicts. Currently, we err on the side of raising a merge conflict rather than deciding to use one version for a certain change, but in practice most conflicts have a pretty obvious resolution when a human looks at the two conflicting versions.
- lewisjoe 7y ago
  > The issue is less about which fancy algorithm/data structure to use and more about even defining in human terms what would be expected in certain merge conflicts Exactly. When it comes to rich text editing which can have semantic trees (e.g tables and tables within tables), merge conflicts are so tough to handle. Consider this: one version deleting a column in a table, and the other version splitting the column and adding a new row - by this time it's sort of impossible to find a meaningful representation of the table without manual intervention.

scofalik 7y ago

> One way to solve this is to fallback to differential sync when syncing huge offline edits This is another question -- does automatic syncing make sense for huge differences in users' contents? The answer might be no. OTOH, isn't this a rare scenario? How perfect it has to be?How much of a compromise can the users accept? These are interesting questions which shows that creating this kind of software isn't easy -- both from technical and UX point of view.

jgtrosh 7y ago
Well it's quite common for people using “normal” email-exchanged MS office revisions. I have no clue if there's specific tooling for these cases. Of course you could brush them off and say they wouldn't be a problem if they just used online editors.
qpiox 7y ago
First of all when discussing about offline edits we should make effort to draw attention that "offline" could mean different thing to different people. Offline editor from the point of view of the app's software developer might mean that editor can sometimes survive few minutes of lost connectivity, after which it will reconnect and possibly sync the team work. Offline from the perspective of a team of scientists writing a joint paper means that some of them will take their work truly offline, to a secluded mountain hut, for an extended period (week or month) and will come back with a complete rewrite of the text to the point of being unrecognizable to the other authors. And then the coauthors will disagree with most of it, and will want to revert some of the pieces and keep some other pieces. And scientists are not the only people needing joint collaborative environment. And not just for short papers. Few years ago, I was working in a larger team with a group of people doing revisions of a book, and there were so many revisions and revisions of revisions of revisions, and zillions of comments, that we had to split the book not in chapters, since Word got stuck even with 20-page chapters. We had to split the chapters in sub-chapters, of 5-10 pages in separate files, so that the processor would not get stuck with the myriad of comments and revisions. These are all separate scenarios in collaborative writing, and there could be many other scenarios, so any solution should first explain for what type of scenario is it really targeting. Automated sync of collaborative work is not always the best thing to do.
- bluGill 7y ago
  Then there is the science fiction version where the editors are both online, but several light years apart with the resulting lag in updates.

bachmeier 7y ago

> sync large offline edits, with the intentions preserved How would that be possible if none of the users can see what the others are doing? If you have three users that start with version 1 and they all make their own version 2, even the users don't know what the "correct" version 2 should look like. Or to put it differently, I don't even know what is meant by "intentions" in this scenario.

baddox 7y ago

I work with a tool that provides diffing and merging for rich text (effectively a limited subset of HTML). It doesn’t have or need real-time collaborative functionality at all. I have read all the papers that I can find and understand, but I’m still not happy with our merging algorithm for rich text. The issue is less about which fancy algorithm/data structure to use and more about even defining in human terms what would be expected in certain merge conflicts. Currently, we err on the side of raising a merge conflict rather than deciding to use one version for a certain change, but in practice most conflicts have a pretty obvious resolution when a human looks at the two conflicting versions.

lewisjoe 7y ago
> The issue is less about which fancy algorithm/data structure to use and more about even defining in human terms what would be expected in certain merge conflicts Exactly. When it comes to rich text editing which can have semantic trees (e.g tables and tables within tables), merge conflicts are so tough to handle. Consider this: one version deleting a column in a table, and the other version splitting the column and adding a new row - by this time it's sort of impossible to find a meaningful representation of the table without manual intervention.