4 ms·

Is this really that novel? I mean using a central process for serializing a distributed system is like a no brainer -- didn't we start off from here originally?

Is this really that novel? I mean using a central process for serializing a distributed system is like a no brainer -- didn't we start off from here originally? -- until you have to worry about network partitions, and CAP and all that jazz. You also now have a single point of failure. Also I skimmed the thing but was performance discussed?

bongodongobob 1y ago
Yeah I have the same question. I'm not familiar with the problem space but this seems like my naive first idea so I'm wondering what the catch is.
- mweidner 1y ago
  As the author, same. My best guess is: - Central-server collaborative editing work focuses on Operational Transformation (OT), likely due to inertia (studied since 1989) and the perception that storing an ID per character is inefficient. In fairness, it is, absent the optimizations introduced by RGASplit and Yjs (~2015). - For decentralized editing, OT is very complicated, and CRDTs took over as the solution of interest (studied since 2005). Storing every operation permanently in a log - needed to use the linked approach without a server - feels inefficient, as does server reconciliation's undo/re-apply process. So CRDT research has focused on avoiding those inefficiencies, sacrificing simplicity along the way, instead of just embracing them as the easy way out. To me, the "inefficiencies" seem quite manageable. Storage is cheap, text is small, and you probably want a complete op log anyway for auditing and document histories (cf. git). Server reconciliation's undo/re-apply process can be batched aggressively, e.g., only do it a few times per second; that just makes remote ops take a little longer to show up. Granted, I have not built a complete app around server reconciliation or the linked approach, so perhaps there is a hidden catch. But I am encouraged by the success of Replicache (https://doc.replicache.dev/concepts/how-it-works https://doc.replicache.dev/concepts/how-it-works), which is where I learned of server reconciliation.
  - kamranjon 1y ago
    I wonder if you could sidestep the inefficiencies of storing ID's for every character by using redis streams to store the characters with id's "represented as delta-compressed macro nodes that are linked together by a radix tree" where the ids are "monotonically incrementing and has two parts: <time>-<counter>. The time is in milliseconds and the counter increases for entries generated in the same milliseconds" This would seem to avoid clashes and also compress the identifier storage in an efficient way. https://antirez.com/news/128 https://antirez.com/news/128
    - mweidner 1y ago
      This sounds similar to the idea behind articulated (though with ids UUID-counter instead of time-counter): https://github.com/mweidner037/articulated https://github.com/mweidner037/articulated I will check out Antirez.
thebeardisred 1y ago
I really want to believe you were trying to make a reference to cap'n jazz - https://en.m.wikipedia.org/wiki/Cap'n_Jazz https://en.m.wikipedia.org/wiki/Cap'n_Jazz :)
- yubblegum 1y ago
  Never heard of them :(
motorest 1y ago
> Is this really that novel? I mean using a central process for serializing a distributed system is like a no brainer -- didn't we start off from here originally? Yes. This article reads like the adage "a month in a lab saves you an hour in a library".

bongodongobob 1y ago

Yeah I have the same question. I'm not familiar with the problem space but this seems like my naive first idea so I'm wondering what the catch is.

mweidner 1y ago
As the author, same. My best guess is: - Central-server collaborative editing work focuses on Operational Transformation (OT), likely due to inertia (studied since 1989) and the perception that storing an ID per character is inefficient. In fairness, it is, absent the optimizations introduced by RGASplit and Yjs (~2015). - For decentralized editing, OT is very complicated, and CRDTs took over as the solution of interest (studied since 2005). Storing every operation permanently in a log - needed to use the linked approach without a server - feels inefficient, as does server reconciliation's undo/re-apply process. So CRDT research has focused on avoiding those inefficiencies, sacrificing simplicity along the way, instead of just embracing them as the easy way out. To me, the "inefficiencies" seem quite manageable. Storage is cheap, text is small, and you probably want a complete op log anyway for auditing and document histories (cf. git). Server reconciliation's undo/re-apply process can be batched aggressively, e.g., only do it a few times per second; that just makes remote ops take a little longer to show up. Granted, I have not built a complete app around server reconciliation or the linked approach, so perhaps there is a hidden catch. But I am encouraged by the success of Replicache (https://doc.replicache.dev/concepts/how-it-works https://doc.replicache.dev/concepts/how-it-works), which is where I learned of server reconciliation.
- kamranjon 1y ago
  I wonder if you could sidestep the inefficiencies of storing ID's for every character by using redis streams to store the characters with id's "represented as delta-compressed macro nodes that are linked together by a radix tree" where the ids are "monotonically incrementing and has two parts: <time>-<counter>. The time is in milliseconds and the counter increases for entries generated in the same milliseconds" This would seem to avoid clashes and also compress the identifier storage in an efficient way. https://antirez.com/news/128 https://antirez.com/news/128
  - mweidner 1y ago
    This sounds similar to the idea behind articulated (though with ids UUID-counter instead of time-counter): https://github.com/mweidner037/articulated https://github.com/mweidner037/articulated I will check out Antirez.

thebeardisred 1y ago

I really want to believe you were trying to make a reference to cap'n jazz - https://en.m.wikipedia.org/wiki/Cap'n_Jazz https://en.m.wikipedia.org/wiki/Cap'n_Jazz :)

yubblegum 1y ago
Never heard of them :(

motorest 1y ago

> Is this really that novel? I mean using a central process for serializing a distributed system is like a no brainer -- didn't we start off from here originally? Yes. This article reads like the adage "a month in a lab saves you an hour in a library".