4 ms·

Really interested to see how this turns out. In addition to solving the number of problems that CRDTs solve when it comes to text, I would like to see examples

by agg23 3y ago

Really interested to see how this turns out. In addition to solving the number of problems that CRDTs solve when it comes to text, I would like to see examples of integrating a CRDT system that doesn't solely have text as its stored asset. For example, say you're storing events alongside your CRDT datastructure (the text). Assuming the events can be blindly merged, what does a P2P sync look like? If it's something that's handled by the CRDT framework (as it looks like Loro will do), do you have to maintain two different sync processes?

codetrotter 3y ago
What kind of events do you have in mind?
- agg23 3y ago
  Naively, literally logs/timeseries events (i.e. stuff that can literally be inserted and queried in order of their timestamp). This scenario can be made increasingly complex as you add user interaction, such as a settings JSON blob that you probably do want to intelligently sync. ---- In general, particularly for the timeseries case, I just want to know what the intended P2P flow looks like for different datatypes, including those that probably aren't managed by the CRDT algorithms. It seems like these systems beeline for the hard text case (which makes sense, because it's the interesting and challenging problem), but forget that in the real world, you need to sync more than just text.
iudqnolq 3y ago
Simple CRDTs are surprisingly simple. If you don't need text the complexity drops off quite quickly. I just ripped yjs out of a personal project and replaced it with something I wrote from scratch. Getting exactly the semantics I want is nice. For example it's easier to make compound properties update atomically. A good place to get started is one of the figma founders' blogs: https://madebyevan.com/algos/crdt-fractional-indexing/ https://madebyevan.com/algos/crdt-fractional-indexing/ You'll notice if you use Figma that it's mostly last-writer-wins fields. That seems to often turn out to be the ideal UI and it's the simplest to implement. If you the text to green and I set it to red the right answer isn't yellow.
mycall 3y ago
I've never need an event-based data flow that doesn't require correct sequencing. Blindly merging is quite a delicate assumption.
- agg23 3y ago
  Sure, there's plenty of complexity there too. For simplicity, I was just assuming timestamps are fine, and we're not concerned about apparent concurrent events or incorrect timestamps from a host.
  - iudqnolq 3y ago
    Your crdt library should be maintaining some kind of logical timestamp. I'd put that in your event and use it to order. That way you get a consistent view of time. (For example yjs calls this a state vector. Might also be called a Lamport timestamp or a vector clock.)

by agg23 3y ago

codetrotter 3y ago

What kind of events do you have in mind?

agg23 3y ago
Naively, literally logs/timeseries events (i.e. stuff that can literally be inserted and queried in order of their timestamp). This scenario can be made increasingly complex as you add user interaction, such as a settings JSON blob that you probably do want to intelligently sync. ---- In general, particularly for the timeseries case, I just want to know what the intended P2P flow looks like for different datatypes, including those that probably aren't managed by the CRDT algorithms. It seems like these systems beeline for the hard text case (which makes sense, because it's the interesting and challenging problem), but forget that in the real world, you need to sync more than just text.

iudqnolq 3y ago

Simple CRDTs are surprisingly simple. If you don't need text the complexity drops off quite quickly. I just ripped yjs out of a personal project and replaced it with something I wrote from scratch. Getting exactly the semantics I want is nice. For example it's easier to make compound properties update atomically. A good place to get started is one of the figma founders' blogs: https://madebyevan.com/algos/crdt-fractional-indexing/ https://madebyevan.com/algos/crdt-fractional-indexing/ You'll notice if you use Figma that it's mostly last-writer-wins fields. That seems to often turn out to be the ideal UI and it's the simplest to implement. If you the text to green and I set it to red the right answer isn't yellow.

mycall 3y ago

I've never need an event-based data flow that doesn't require correct sequencing. Blindly merging is quite a delicate assumption.

agg23 3y ago
Sure, there's plenty of complexity there too. For simplicity, I was just assuming timestamps are fine, and we're not concerned about apparent concurrent events or incorrect timestamps from a host.
- iudqnolq 3y ago
  Your crdt library should be maintaining some kind of logical timestamp. I'd put that in your event and use it to order. That way you get a consistent view of time. (For example yjs calls this a state vector. Might also be called a Lamport timestamp or a vector clock.)