2 ms·

Diamond types author here! Congratulations on getting your crdt working! It’s lovely to see a new generation of CRDTs which have decent performance. And nice s

Diamond types author here! Congratulations on getting your crdt working! It’s lovely to see a new generation of CRDTs which have decent performance. And nice stuff implementing peritext! I’d love to do the same in diamond types at some point. You beat me to it! Im building a little repository of real world collaborative editing traces to use when benchmarking, comparing and optimising text based CRDTs[1]. The automerge-perf editing trace isn’t enough on its own. And we’re increasingly converging on a format for multi user concurrent editing traces too[2]. It’d be great to add some rich text editing traces in the mix if you’re interested in recording something, so we can also compare how peritext performs in different systems. Anyway, welcome to the community! Love to have more implementations around! https://github.com/josephg/crdt-benchmarks https://github.com/josephg/crdt-benchmarks https://github.com/dmonad/crdt-benchmarks/issues/20 https://github.com/dmonad/crdt-benchmarks/issues/20

czx111331 3y ago
Hi! I really like your work, and I've learned a lot from your code ;) I totally agree that we need more real-world datasets to make better benchmarks and avoid 'overfitting'. It'd be nice to have benchmarks for rich text. I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this?
- josephg 3y ago
  Thanks!! I was a bit disappointed diamond types isn't in the table of benchmarking results. Mind if I send you a PR adding it? I also think it'd be nice to have a native rust benchmarking suite for this stuff. All of the CRDT libraries are either in rust or are moving to rust. Some benchmarks with criterion would be nice to see. > I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this? Thats been on my todo list for awhile. I love performance optimization but you can't optimize what you can't measure. I'm just not sure where to get editing traces at the moment given not much software works like this. We could reverse engineer figma enough to record editing traces out of it (or use its plugin API?). Or record some traces from tldraw. Ideally we'd have a few traces from different applications - since I wouldn't be surprised if different apps depend on this stuff differently. Are you intending to add JSON-style editing support to your CRDT? If anyone reading this is working on a collaborative app, and wants your app to be exceptionally well supported by CRDTs - please get in touch and we can talk about getting some editing traces. We can't properly optimize collaborative editing systems without them!
  - czx111331 3y ago
    PRs are welcome! I wasn't aware that Diamond types also provided a WASM version. I've included it in the native benchmark with Loro though. You can see it here: https://www.loro.dev/docs/performance/native https://www.loro.dev/docs/performance/native > Are you intending to add JSON-style editing support to your CRDT? Loro is another CRDT library I'm working on. It's a superset of crdt-richtext and supports JSON-style editing. > We can't properly optimize collaborative editing systems without them! I'm totally with it

czx111331 3y ago

Hi! I really like your work, and I've learned a lot from your code ;) I totally agree that we need more real-world datasets to make better benchmarks and avoid 'overfitting'. It'd be nice to have benchmarks for rich text. I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this?

josephg 3y ago
Thanks!! I was a bit disappointed diamond types isn't in the table of benchmarking results. Mind if I send you a PR adding it? I also think it'd be nice to have a native rust benchmarking suite for this stuff. All of the CRDT libraries are either in rust or are moving to rust. Some benchmarks with criterion would be nice to see. > I've also noticed that certain scenarios, like drawing, table editing, and whiteboard editing, aren't well covered. What are your thoughts on this? Thats been on my todo list for awhile. I love performance optimization but you can't optimize what you can't measure. I'm just not sure where to get editing traces at the moment given not much software works like this. We could reverse engineer figma enough to record editing traces out of it (or use its plugin API?). Or record some traces from tldraw. Ideally we'd have a few traces from different applications - since I wouldn't be surprised if different apps depend on this stuff differently. Are you intending to add JSON-style editing support to your CRDT? If anyone reading this is working on a collaborative app, and wants your app to be exceptionally well supported by CRDTs - please get in touch and we can talk about getting some editing traces. We can't properly optimize collaborative editing systems without them!
- czx111331 3y ago
  PRs are welcome! I wasn't aware that Diamond types also provided a WASM version. I've included it in the native benchmark with Loro though. You can see it here: https://www.loro.dev/docs/performance/native https://www.loro.dev/docs/performance/native > Are you intending to add JSON-style editing support to your CRDT? Loro is another CRDT library I'm working on. It's a superset of crdt-richtext and supports JSON-style editing. > We can't properly optimize collaborative editing systems without them! I'm totally with it