6 ms·

Real-time, collaborative Markdown editor with end-to-end encryption

194 points by shrikant 10y ago

mobitar 10y ago
Here's a sample document: https://extensions.standardnotes.org/collab/doc/741ec80a-3667-46d4-b94d-6621fc2bf265#key=5e2b16147d1b344628b0e1eeb57219c97b4099d918ae63549685dbe00a2ea548 https://extensions.standardnotes.org/collab/doc/741ec80a-366... Source is available here: https://github.com/standardnotes/collab-editor https://github.com/standardnotes/collab-editor The editor relies on an impressive client-side library called ChainPad, which uses blockchains as inspiration for determining the authoritative document after conflicts or many simultaneous edits. Typically operational transformation algorithms and systems to manage conflicts are handled by the server, precluding the possibility for end-to-end encryption. However, ChainPad runs completely on the client-side, and is oblivious to the underlying text, thus allowing us to encrypt text client-side before broadcasting to other participants and the broadcast server. This is the first major effort I've seen for a real time client-side collaboration algorithm, and its use of a blockchain type structure is ingenious. More info on ChainPad here: https://github.com/xwiki-contrib/chainpad https://github.com/xwiki-contrib/chainpad
- josephg 10y ago
  Thanks for the explanation. Looks like its not just E2E encrypted but distributed as well. Very cool work! However, you shouldn't need a blockchain to make a fully encrypted realtime collaborative editor. Here the blockchain is needed to make OT work properly because OT relies on a global ordering of operations. But algorithmically you can get around that either using CRDTs or modifying OT with a purge() function. (This lets you move between any valid linearisation of history, so you can transform by rewriting your history to look like the remote client before accepting their operation.) I've been messing around with OT stuff for a few years. Feel free to pick my brain if you have any questions - I'm looking forward to seeing where you go with it! (Grab my email from a commit on github)
  - simplify 10y ago
    What's your opinion of ChainPad's algorithm? Are there any obvious downsides?
    - espadrine 10y ago
      Intention preservation. Convergence, they get from the blockchain. So everyone eventually reaches the same content. But that content may delete a whole paragraph you just wrote, or duplicate it, or do any number of monstrous things, while your wifi was flaky. Make this test: open a document in two tabs, write "my test", go offline, write "is done" at the end in one, and delete "my" in the other, go online, wait for synchronization. You may see: "my test is donetestmy test i". Intention not preserved. (Just to be clear: this isn't just a problem for offline; network latency is dangerous.) So local changes (not yet sent to others) are rebased with an OT-style algorithm, although a seemingly faulty one. (OT is brutal to implement.) But it gets worse: > If a Patch is rooted in a previous state of the document which is not the Authoritative Document, the patch is stored in case it might be part of a fork of the patch-chain which proves longer than the chain which the engine currently is aware of. A blockchain is a type of Merkle tree. When the tree has two children, only the longest chain matters. So, you lose your work if someone forks the chain you were on and makes a ton of edits. Admittedly, you wouldn't give edit access to an untrusted party anyway. But that merely makes the security guarantees of the blockchain irrelevant! Reaching convergence is easy: you simply need total order. Use a Lamport timestamp and encrypt patches with your private key, and you get the same guarantees for a fraction of the cost and complexity of chainpad. If you want intention preservation too, there has been great efforts in OT to ensure the operations you make are applied in their intended way, so no deletion or duplication of content. (CRDT actually gets it for free, along with peer-to-peer, lucky them.) End-to-end encryption is not incompatible with either OT or CRDT; just encrypt your patches locally, and others can decrypt them with your public key. (It's probably much, much easier to implement with CRDT though, just because peer-to-peer OT is not trivial.) (Having read this, it probably comes across as more aggressive than planned. I am actually fairly impressed by chainpad: it is a fun and ambitious project. But I wouldn't trust it with data I don't want to lose.)
- eps 10y ago
  > Here's a sample document Looks broken on Mobile Safari. "Viewing URL" and "Editing URL" are both empty, there's a blank two-line textarea and that's about it.
akkartik 10y ago
I recently came across CryptPad: https://news.ycombinator.com/item?id=12566326 https://news.ycombinator.com/item?id=12566326
- mobitar 10y ago
  Yup, this uses ChainPad as its underlying algorithm, which is made by the same guys at CryptPad.
franciscop 10y ago
The back button seems broken so you might want to check on the History API. I understand you need a pure JS redirect for generating the encrypted URL so that would seem like the best way to go. Otherwise you could also do: - Add a button in the main page that generates the page in-the-spot. - Add a button in the middle page that generates the page when clicked. Edit: removed dupe link so added more content
tracker1 10y ago
Nice... though would be cool to have an integrated preview on the right-side of the screen.
- pmontra 10y ago
  I was about to ask for the same feature. A download link would be enough to start with, to open the text with Typora or other viewers (Typora is an editor.) I looked at the Android app. It asks for access to contacts and accounts. I don't like that but maybe I'll use it with my own server (Rail 5, great for me!) Still, why does it need those permissions? Anyway, I'm on Android 7 so I can probably not allow them when asked. (I'm not able to fork and remove them, not in Android development.)
  - mobitar 10y ago
    Whoops, shouldn't be asking for those permissions. We're fixing this now.
cyphar 10y ago
> Since the encryption key follows the # symbol, it is never sent to the server. This doesn't seem like a safe idea to me. While it's true that browsers don't send the # part of a URL when fetching a page, you can use JavaScript to get the value (in fact that's what this webapp is doing). So what is the mitigation if someone manages to XSS the website and start snarfing the encryption keys for everyone's messages? Really cool idea BTW.
- chii 10y ago
  if someone already can XSS your site, then your site is already owned, and there's no need to sniff the key!
  - MildlySerious 10y ago
    Plus, no matter how else you would store the key, it could be retrieved just as easily if the site is compromised on the frontend.
nathancahill 10y ago
If anyone wants to play around with it: https://extensions.standardnotes.org/collab/doc/47b5d1f3-e26f-4be8-9438-418a0f9c5fd8?et=524f7fb1727d682ae1a8167d#key=d6a0e729ecfc1101a9cc02273d476704a96959cdf9b518e2387ab711abbc2e87 https://extensions.standardnotes.org/collab/doc/47b5d1f3-e26...
KirinDave 10y ago
I'm a bit confused about ChainPad. Why use a blockchain here? Wouldn't treedoc be enormously better? I confess to being a bit uninformed about how chainpad resolves editing being done in the exact same area by multiple people all at once. Treedoc handles this elegantly and with good performance. Does ChainPad? I'd test myself, but since I lack friends it's difficult to do correctly.
chuckdries 10y ago
Wow I've been using Standard Notes for my normal note taking for a while now. Cool stuff.
grogenaut 10y ago
One thing that scares me is that links like they should be shared. So I think people would just be sending these links to the other person over open email. But the link IS the doc, and people will share it incorrectly. So it doesn't hit the server with the doc on it... but it hits facebook, gmail or snapchat so the other person can get it.
- mobitar 10y ago
  This tool isn't targeted at Snapchat users ;)
- ldubost 10y ago
  (I'm part of the cryptpad team) This is indeed an issue, and on cryptpad we will one to tackle that issue in the future with other ways to share the key that does not involve copying it on unsecure channels. For now there is though a difference between dropping you URL in unsecure channels and having your content lying around unsecured at a hoster's site.. the first difference is that your hoster has your content but might not be able to read your snapchat
  - grogenaut 10y ago
    Putting my security hat on, why are you worried about the host being the attack vector more than in transit? The host could easily alter the javascript that is sent back to sniff the crypto token.
chuckdries 10y ago
Wow I've been using Standard Notes for my normal note taking for a while now. Cool stuff.
Kubuxu 10y ago
It uses the same base as Cryptpad [0] (Chainpad) which is created by the same people that created Chainpad. It has more functions like a presentation mode [1]. The edit resolution is done client side and the "server" only relays encrypted messages. Alternative transports are in works, including WebRTC. Everything is open source on [2]. [0]: https://cryptpad.fr https://cryptpad.fr [1]: https://cryptpad.fr/slide/#/1/view/xQOAr26XzkbKDNuXvXwL4Q/E-YYyKn8sKRCPUXdn73s3kOHCwq25GOjx3DH9V1n6MY/present https://cryptpad.fr/slide/#/1/view/xQOAr26XzkbKDNuXvXwL4Q/E-... [2]: https://github.com/xwiki-labs/cryptpad/ https://github.com/xwiki-labs/cryptpad/
k__ 10y ago
Wanted to write such a thing for AsciiDoc once. Cool idea :)

194 points by shrikant 10y ago

Here's a sample document: https://extensions.standardnotes.org/collab/doc/741ec80a-3667-46d4-b94d-6621fc2bf265#key=5e2b16147d1b344628b0e1eeb57219c97b4099d918ae63549685dbe00a2ea548 https://extensions.standardnotes.org/collab/doc/741ec80a-366... Source is available here: https://github.com/standardnotes/collab-editor https://github.com/standardnotes/collab-editor The editor relies on an impressive client-side library called ChainPad, which uses blockchains as inspiration for determining the authoritative document after conflicts or many simultaneous edits. Typically operational transformation algorithms and systems to manage conflicts are handled by the server, precluding the possibility for end-to-end encryption. However, ChainPad runs completely on the client-side, and is oblivious to the underlying text, thus allowing us to encrypt text client-side before broadcasting to other participants and the broadcast server. This is the first major effort I've seen for a real time client-side collaboration algorithm, and its use of a blockchain type structure is ingenious. More info on ChainPad here: https://github.com/xwiki-contrib/chainpad https://github.com/xwiki-contrib/chainpad

josephg 10y ago
Thanks for the explanation. Looks like its not just E2E encrypted but distributed as well. Very cool work! However, you shouldn't need a blockchain to make a fully encrypted realtime collaborative editor. Here the blockchain is needed to make OT work properly because OT relies on a global ordering of operations. But algorithmically you can get around that either using CRDTs or modifying OT with a purge() function. (This lets you move between any valid linearisation of history, so you can transform by rewriting your history to look like the remote client before accepting their operation.) I've been messing around with OT stuff for a few years. Feel free to pick my brain if you have any questions - I'm looking forward to seeing where you go with it! (Grab my email from a commit on github)
- simplify 10y ago
  What's your opinion of ChainPad's algorithm? Are there any obvious downsides?
  - espadrine 10y ago
    Intention preservation. Convergence, they get from the blockchain. So everyone eventually reaches the same content. But that content may delete a whole paragraph you just wrote, or duplicate it, or do any number of monstrous things, while your wifi was flaky. Make this test: open a document in two tabs, write "my test", go offline, write "is done" at the end in one, and delete "my" in the other, go online, wait for synchronization. You may see: "my test is donetestmy test i". Intention not preserved. (Just to be clear: this isn't just a problem for offline; network latency is dangerous.) So local changes (not yet sent to others) are rebased with an OT-style algorithm, although a seemingly faulty one. (OT is brutal to implement.) But it gets worse: > If a Patch is rooted in a previous state of the document which is not the Authoritative Document, the patch is stored in case it might be part of a fork of the patch-chain which proves longer than the chain which the engine currently is aware of. A blockchain is a type of Merkle tree. When the tree has two children, only the longest chain matters. So, you lose your work if someone forks the chain you were on and makes a ton of edits. Admittedly, you wouldn't give edit access to an untrusted party anyway. But that merely makes the security guarantees of the blockchain irrelevant! Reaching convergence is easy: you simply need total order. Use a Lamport timestamp and encrypt patches with your private key, and you get the same guarantees for a fraction of the cost and complexity of chainpad. If you want intention preservation too, there has been great efforts in OT to ensure the operations you make are applied in their intended way, so no deletion or duplication of content. (CRDT actually gets it for free, along with peer-to-peer, lucky them.) End-to-end encryption is not incompatible with either OT or CRDT; just encrypt your patches locally, and others can decrypt them with your public key. (It's probably much, much easier to implement with CRDT though, just because peer-to-peer OT is not trivial.) (Having read this, it probably comes across as more aggressive than planned. I am actually fairly impressed by chainpad: it is a fun and ambitious project. But I wouldn't trust it with data I don't want to lose.)
eps 10y ago
> Here's a sample document Looks broken on Mobile Safari. "Viewing URL" and "Editing URL" are both empty, there's a blank two-line textarea and that's about it.

akkartik 10y ago

I recently came across CryptPad: https://news.ycombinator.com/item?id=12566326 https://news.ycombinator.com/item?id=12566326

mobitar 10y ago
Yup, this uses ChainPad as its underlying algorithm, which is made by the same guys at CryptPad.

franciscop 10y ago

The back button seems broken so you might want to check on the History API. I understand you need a pure JS redirect for generating the encrypted URL so that would seem like the best way to go. Otherwise you could also do: - Add a button in the main page that generates the page in-the-spot. - Add a button in the middle page that generates the page when clicked. Edit: removed dupe link so added more content

tracker1 10y ago

Nice... though would be cool to have an integrated preview on the right-side of the screen.

pmontra 10y ago
I was about to ask for the same feature. A download link would be enough to start with, to open the text with Typora or other viewers (Typora is an editor.) I looked at the Android app. It asks for access to contacts and accounts. I don't like that but maybe I'll use it with my own server (Rail 5, great for me!) Still, why does it need those permissions? Anyway, I'm on Android 7 so I can probably not allow them when asked. (I'm not able to fork and remove them, not in Android development.)
- mobitar 10y ago
  Whoops, shouldn't be asking for those permissions. We're fixing this now.

cyphar 10y ago

> Since the encryption key follows the # symbol, it is never sent to the server. This doesn't seem like a safe idea to me. While it's true that browsers don't send the # part of a URL when fetching a page, you can use JavaScript to get the value (in fact that's what this webapp is doing). So what is the mitigation if someone manages to XSS the website and start snarfing the encryption keys for everyone's messages? Really cool idea BTW.

chii 10y ago
if someone already can XSS your site, then your site is already owned, and there's no need to sniff the key!
- MildlySerious 10y ago
  Plus, no matter how else you would store the key, it could be retrieved just as easily if the site is compromised on the frontend.

nathancahill 10y ago

If anyone wants to play around with it: https://extensions.standardnotes.org/collab/doc/47b5d1f3-e26f-4be8-9438-418a0f9c5fd8?et=524f7fb1727d682ae1a8167d#key=d6a0e729ecfc1101a9cc02273d476704a96959cdf9b518e2387ab711abbc2e87 https://extensions.standardnotes.org/collab/doc/47b5d1f3-e26...

KirinDave 10y ago

I'm a bit confused about ChainPad. Why use a blockchain here? Wouldn't treedoc be enormously better? I confess to being a bit uninformed about how chainpad resolves editing being done in the exact same area by multiple people all at once. Treedoc handles this elegantly and with good performance. Does ChainPad? I'd test myself, but since I lack friends it's difficult to do correctly.

chuckdries 10y ago

Wow I've been using Standard Notes for my normal note taking for a while now. Cool stuff.

grogenaut 10y ago

One thing that scares me is that links like they should be shared. So I think people would just be sending these links to the other person over open email. But the link IS the doc, and people will share it incorrectly. So it doesn't hit the server with the doc on it... but it hits facebook, gmail or snapchat so the other person can get it.

mobitar 10y ago
This tool isn't targeted at Snapchat users ;)
ldubost 10y ago
(I'm part of the cryptpad team) This is indeed an issue, and on cryptpad we will one to tackle that issue in the future with other ways to share the key that does not involve copying it on unsecure channels. For now there is though a difference between dropping you URL in unsecure channels and having your content lying around unsecured at a hoster's site.. the first difference is that your hoster has your content but might not be able to read your snapchat
- grogenaut 10y ago
  Putting my security hat on, why are you worried about the host being the attack vector more than in transit? The host could easily alter the javascript that is sent back to sniff the crypto token.

chuckdries 10y ago

Wow I've been using Standard Notes for my normal note taking for a while now. Cool stuff.

Kubuxu 10y ago

It uses the same base as Cryptpad [0] (Chainpad) which is created by the same people that created Chainpad. It has more functions like a presentation mode [1]. The edit resolution is done client side and the "server" only relays encrypted messages. Alternative transports are in works, including WebRTC. Everything is open source on [2]. [0]: https://cryptpad.fr https://cryptpad.fr [1]: https://cryptpad.fr/slide/#/1/view/xQOAr26XzkbKDNuXvXwL4Q/E-YYyKn8sKRCPUXdn73s3kOHCwq25GOjx3DH9V1n6MY/present https://cryptpad.fr/slide/#/1/view/xQOAr26XzkbKDNuXvXwL4Q/E-... [2]: https://github.com/xwiki-labs/cryptpad/ https://github.com/xwiki-labs/cryptpad/

k__ 10y ago

Wanted to write such a thing for AsciiDoc once. Cool idea :)