5 ms·

Hmm, it feels like a CRDT + any overlay network (This is probably how you use tor or IPFS p2p) would get you there. Matrix p2p is developing pinecone [1] as an

Hmm, it feels like a CRDT + any overlay network (This is probably how you use tor or IPFS p2p) would get you there. Matrix p2p is developing pinecone [1] as an overlay network, which seems very aligned with your needs, though, as you say, it is less mature. Well, pinecone/yggdrasil/cjdns/tor are only part of the question, as they provide encrypted E2E connectivity. You still have to build on top of it to handle groups and lazy loading. They do solve routing though, so you can just try to send packets to every group member (like Matrix does). For a more efficient implementation, it would be nice to have multicast/unicast integrated in the overlay network at the protocol level. API-wise, it could be made as simple as sharing a private key with a group: multiple peers with the same address would then receive the same packets. In any case, I feel like working together with Matrix (/new vector) on pinecone would help both of you. [1]: https://github.com/matrix-org/pinecone https://github.com/matrix-org/pinecone

holmesworcester 3y ago
I've been following the P2P Matrix work intermittently but closely. Matrix clients are so much more mature than ours, so having a first-class P2P mode in Matrix would be a solution to this whole problem. That said, they have to solve more problems than just the problems solved by Pinecone. (Like onboarding and identity.) And over the past few years my impression has been that the Matrix team is focused on the federated use case, because that's where their users are. It seems hard for them to pay attention to both. My guess is that P2P Matrix will land as an awesome reliability feature for uninterrupted messaging when a server goes down or when the Internet goes out, but that it will take longer for them to tackle problems specific to P2P that don't graft on easily to federated Matrix.
- Arathorn 3y ago
  The reason that P2P Matrix (arewep2pyet.com) exists is precisely to provide a 1st class P2P mode to Matrix - it's not just for awesome reliability and network resilience, but for easier onboarding; killing off public-service homeservers like matrix.org; etc. The onboarding and identity piece is covered by MSC4014 (which just got implemented in Dendrite.) However, you're totally right that our main focus is on the federated use case, because that's where everyone is today, and its own fair share of Hard Problems (e.g. reliable decentralised E2EE, byzantine-fault-tolerant conversation replication, decentralised ACLs, lazy-loaded conversation state) before we chuck P2P into the mix too. The P2P Matrix project is currently focusing mainly on MSC4014 (i.e. pseudo-IDs, account portability, multihomed accounts) which benefits both P2P and normal Matrix... although I'm hoping that we'll get back to Pinecone & actual P2P work eventually (especially if someone explicitly funds it). Meanwhile, it's super cool to see the progress on Quiet and OrbitDB. It's come a long way since Orbit launched at the 2016 Decentralised Web Summit, and I'm glad to see the tech is still progressing. FWIW, We did the first versions of P2P Matrix on top of libp2p (but then switched to Yggdrasil and then Pinecone in order to work with a monolithic stack, for better or worse). Might be interesting to figure out how the projects could interop in future :)
  - eTomte 3y ago
    I'm a Matrix enthusiast, run my own homeserver and I've been playing around with Quiet today. It has nowhere near the features of Matrix at this point, but it is useable and I'm having fun with it, and I really look forward to see progress on both projects. :)
    - holmesworcester 3y ago
      Thanks for the interest and kind words!
  - holmesworcester 3y ago
    This is all great news, and it's really cool to get an update on the state of P2P Matrix from its creators deep in a thread like this. HN is amazing. Re: interop, the easiest way to interop would be if Matrix solves all these problems for us before we get anywhere close! Then we can be a P2P Matrix client or just use Matrix :) But I bet either way it will be possible to make robust transformation layers between different eventually consistent protocols. This is similar to one of the hard problems: interop between multiple versions of a single decentralized application. If we solve that it might generalize if we can bridge the network layers too? And yes, there is indeed a really long list of hard problems to solve. I really really hope one of us, or one of the other similarly motivated projects, is able to solve them all and deliver the whole package!!!

holmesworcester 3y ago

I've been following the P2P Matrix work intermittently but closely. Matrix clients are so much more mature than ours, so having a first-class P2P mode in Matrix would be a solution to this whole problem. That said, they have to solve more problems than just the problems solved by Pinecone. (Like onboarding and identity.) And over the past few years my impression has been that the Matrix team is focused on the federated use case, because that's where their users are. It seems hard for them to pay attention to both. My guess is that P2P Matrix will land as an awesome reliability feature for uninterrupted messaging when a server goes down or when the Internet goes out, but that it will take longer for them to tackle problems specific to P2P that don't graft on easily to federated Matrix.

Arathorn 3y ago
The reason that P2P Matrix (arewep2pyet.com) exists is precisely to provide a 1st class P2P mode to Matrix - it's not just for awesome reliability and network resilience, but for easier onboarding; killing off public-service homeservers like matrix.org; etc. The onboarding and identity piece is covered by MSC4014 (which just got implemented in Dendrite.) However, you're totally right that our main focus is on the federated use case, because that's where everyone is today, and its own fair share of Hard Problems (e.g. reliable decentralised E2EE, byzantine-fault-tolerant conversation replication, decentralised ACLs, lazy-loaded conversation state) before we chuck P2P into the mix too. The P2P Matrix project is currently focusing mainly on MSC4014 (i.e. pseudo-IDs, account portability, multihomed accounts) which benefits both P2P and normal Matrix... although I'm hoping that we'll get back to Pinecone & actual P2P work eventually (especially if someone explicitly funds it). Meanwhile, it's super cool to see the progress on Quiet and OrbitDB. It's come a long way since Orbit launched at the 2016 Decentralised Web Summit, and I'm glad to see the tech is still progressing. FWIW, We did the first versions of P2P Matrix on top of libp2p (but then switched to Yggdrasil and then Pinecone in order to work with a monolithic stack, for better or worse). Might be interesting to figure out how the projects could interop in future :)
- eTomte 3y ago
  I'm a Matrix enthusiast, run my own homeserver and I've been playing around with Quiet today. It has nowhere near the features of Matrix at this point, but it is useable and I'm having fun with it, and I really look forward to see progress on both projects. :)
  - holmesworcester 3y ago
    Thanks for the interest and kind words!
- holmesworcester 3y ago
  This is all great news, and it's really cool to get an update on the state of P2P Matrix from its creators deep in a thread like this. HN is amazing. Re: interop, the easiest way to interop would be if Matrix solves all these problems for us before we get anywhere close! Then we can be a P2P Matrix client or just use Matrix :) But I bet either way it will be possible to make robust transformation layers between different eventually consistent protocols. This is similar to one of the hard problems: interop between multiple versions of a single decentralized application. If we solve that it might generalize if we can bridge the network layers too? And yes, there is indeed a really long list of hard problems to solve. I really really hope one of us, or one of the other similarly motivated projects, is able to solve them all and deliver the whole package!!!