4 ms·

Building a collaborative pixel art editor with CRDTs

159 points by jakelazaroff 3y ago

smusamashah 3y ago
I thought it was going to be about this https://ourworldofpixels.com/ https://ourworldofpixels.com/
- dang 3y ago
  Discussed here btw: Our World of Pixels - https://news.ycombinator.com/item?id=36861302 https://news.ycombinator.com/item?id=36861302 - July 2023 (39 comments)
omarfarooq 3y ago
Is the current codebase limited to black pixels?
- jakelazaroff 3y ago
  If you click the color input you should be able to change the color!
pvh 3y ago
What fun, jake! I am really enjoying your series here. When we built Pixelpusher (https://inkandswitch.com/pixelpusher https://inkandswitch.com/pixelpusher) that was the beginning of a long and ongoing journey into the user experience implications of CRDTs. For instance: * how and when should we merge changes? * what if we change our minds later? * how do we know if our data is “in sync” with another user * what does it mean to depend on another change? And so on. Pixelpusher is awfully naive by the standards of today but I have very fond memories of working on it with Jeff Peterson, Jim Pick, and Orion Henry.
adityaathalye 3y ago
See also: discussion on the earlier post; "An interactive intro to CRDTs". https://news.ycombinator.com/item?id=37764581 https://news.ycombinator.com/item?id=37764581
antirez 3y ago
For this kind of stuff (collaborative editing in general), it is probably possible to train a NN model in order to guess what is the best blend between the user conflicts. Something like that would likely work very well, reducing the importance of CRDTs by a lot in all the systems where conflicts are due to humans working at the same thing and where the outcome of the conflict, whatever it is, can't create any damage.
- reaperman 3y ago
  What is the specific problem(s) with CRDT which this novel method would improve?
  - Retr0id 3y ago
    CRDTs give you a tool to resolve conflicts at the data layer, but not necessarily at the semantic layer. In the example, we have a canvas. Turn up the latency. User A draws an outline of a heart on their canvas, and User B draws a smiley face on theirs. After the "network" catches up, the CRDT does it's thing, and our states have been synchronised. But we're left with an overlapping heart/smiley combo that neither collaborator really intended. A really smart system might say "user A started drawing their heart a moment before User B started drawing a smiley. Even though most of the drawing happened concurrently, we will give user A's drawing the priority" (perhaps also making a backup of B's work so that it isn't lost). Resolving conflicts in this way requires an understanding of semantics and intent - that a NN could perhaps provide a heuristic for.
    - antirez 3y ago
      Exactly what I had in mind. Thank you for developing the idea and make it more clear.
    - vsareto 3y ago
      It's hard for me to ignore that generative AI can do images and text so the immediate usefulness of this resolving conflicting edits is going over my head. Both users would have to share an intent that has more details that what a particular conflict and the current canvases have to offer. From there though, you could synthesize a compromise, like let's say choosing a color that isn't either user's first choice but is close enough to both, and also happens to be a good choice given the rest of the image. Or it decides that a compromise isn't a good idea and that choosing one of the user's edits is best. If you didn't want to share that intent though, you are giving up your intent to the machine's intent (however it was trained), so you may as well generate two separate things and use some other tools to mix them.
binarymax 3y ago
Just a side note: I love the syntax highlighter you use. The cursive fonts for attrs and comments are really nice.
- jakelazaroff 3y ago
  Thank you!
- CharlesW 3y ago
  In case anyone else is curious, the font family is Cartograph CF ($75).

159 points by jakelazaroff 3y ago

smusamashah 3y ago

I thought it was going to be about this https://ourworldofpixels.com/ https://ourworldofpixels.com/

dang 3y ago
Discussed here btw: Our World of Pixels - https://news.ycombinator.com/item?id=36861302 https://news.ycombinator.com/item?id=36861302 - July 2023 (39 comments)

omarfarooq 3y ago

Is the current codebase limited to black pixels?

jakelazaroff 3y ago
If you click the color input you should be able to change the color!

pvh 3y ago

What fun, jake! I am really enjoying your series here. When we built Pixelpusher (https://inkandswitch.com/pixelpusher https://inkandswitch.com/pixelpusher) that was the beginning of a long and ongoing journey into the user experience implications of CRDTs. For instance: * how and when should we merge changes? * what if we change our minds later? * how do we know if our data is “in sync” with another user * what does it mean to depend on another change? And so on. Pixelpusher is awfully naive by the standards of today but I have very fond memories of working on it with Jeff Peterson, Jim Pick, and Orion Henry.

adityaathalye 3y ago

See also: discussion on the earlier post; "An interactive intro to CRDTs". https://news.ycombinator.com/item?id=37764581 https://news.ycombinator.com/item?id=37764581

antirez 3y ago

For this kind of stuff (collaborative editing in general), it is probably possible to train a NN model in order to guess what is the best blend between the user conflicts. Something like that would likely work very well, reducing the importance of CRDTs by a lot in all the systems where conflicts are due to humans working at the same thing and where the outcome of the conflict, whatever it is, can't create any damage.

reaperman 3y ago
What is the specific problem(s) with CRDT which this novel method would improve?
- Retr0id 3y ago
  CRDTs give you a tool to resolve conflicts at the data layer, but not necessarily at the semantic layer. In the example, we have a canvas. Turn up the latency. User A draws an outline of a heart on their canvas, and User B draws a smiley face on theirs. After the "network" catches up, the CRDT does it's thing, and our states have been synchronised. But we're left with an overlapping heart/smiley combo that neither collaborator really intended. A really smart system might say "user A started drawing their heart a moment before User B started drawing a smiley. Even though most of the drawing happened concurrently, we will give user A's drawing the priority" (perhaps also making a backup of B's work so that it isn't lost). Resolving conflicts in this way requires an understanding of semantics and intent - that a NN could perhaps provide a heuristic for.
  - antirez 3y ago
    Exactly what I had in mind. Thank you for developing the idea and make it more clear.
  - vsareto 3y ago
    It's hard for me to ignore that generative AI can do images and text so the immediate usefulness of this resolving conflicting edits is going over my head. Both users would have to share an intent that has more details that what a particular conflict and the current canvases have to offer. From there though, you could synthesize a compromise, like let's say choosing a color that isn't either user's first choice but is close enough to both, and also happens to be a good choice given the rest of the image. Or it decides that a compromise isn't a good idea and that choosing one of the user's edits is best. If you didn't want to share that intent though, you are giving up your intent to the machine's intent (however it was trained), so you may as well generate two separate things and use some other tools to mix them.

binarymax 3y ago

Just a side note: I love the syntax highlighter you use. The cursive fonts for attrs and comments are really nice.

jakelazaroff 3y ago
Thank you!
CharlesW 3y ago
In case anyone else is curious, the font family is Cartograph CF ($75).