4 ms·

What about Veilid?

by tyzoid 3y ago

What about Veilid?

holmesworcester 3y ago
Veilid is a recently announced, general-purpose peer-to-peer Stack. We're excited about it! We would consider switching away from OrbitDB/IPFS/Libp2p/Tor for a less mature peer-to-peer state syncing stack if it offered the following as first-class features in addition to what we have, had credible longevity, and ran well on desktop and mobile: 1. Deletion 2. Encrypted groups with removal 3. Support for linking multiple devices, and removing devices 4. Support for partial syncing / lazy loading. It would be really nice to have all of these things out of the box, alongside state syncing and BitTorrent-style file transfer (which we have now.) Veilid is still new so we haven't assessed it yet, but that would be our criteria. My general perspective on general purpose peer-to-peer stacks (which I understand Veilid to be) is that it will be very difficult to build meaningful general purpose platforms until we have more examples of popular desktop + mobile peer-to-peer products with large numbers of users in the wild that would let us identify all the things a general purpose stack would need to have, and that right now the best thing to do is build single-purpose peer-to-peer apps in whatever way you can (whether it's by building your own non-general stack or cobbling together existing building blocks as we're doing) and struggle to find product market fit! The way I think about it is: would you have been able to build Heroku before there were a few successful production Ruby on Rails apps with product-market fit? Probably not! So we shouldn't expect to be able to build useful general purpose p2p stacks without clear examples of successful p2p products with product-market fit.
- MayeulC 3y ago
  Hmm, it feels like a CRDT + any overlay network (This is probably how you use tor or IPFS p2p) would get you there. Matrix p2p is developing pinecone [1] as an overlay network, which seems very aligned with your needs, though, as you say, it is less mature. Well, pinecone/yggdrasil/cjdns/tor are only part of the question, as they provide encrypted E2E connectivity. You still have to build on top of it to handle groups and lazy loading. They do solve routing though, so you can just try to send packets to every group member (like Matrix does). For a more efficient implementation, it would be nice to have multicast/unicast integrated in the overlay network at the protocol level. API-wise, it could be made as simple as sharing a private key with a group: multiple peers with the same address would then receive the same packets. In any case, I feel like working together with Matrix (/new vector) on pinecone would help both of you. [1]: https://github.com/matrix-org/pinecone https://github.com/matrix-org/pinecone
  - holmesworcester 3y ago
    I've been following the P2P Matrix work intermittently but closely. Matrix clients are so much more mature than ours, so having a first-class P2P mode in Matrix would be a solution to this whole problem. That said, they have to solve more problems than just the problems solved by Pinecone. (Like onboarding and identity.) And over the past few years my impression has been that the Matrix team is focused on the federated use case, because that's where their users are. It seems hard for them to pay attention to both. My guess is that P2P Matrix will land as an awesome reliability feature for uninterrupted messaging when a server goes down or when the Internet goes out, but that it will take longer for them to tackle problems specific to P2P that don't graft on easily to federated Matrix.
    - Arathorn 3y ago
      The reason that P2P Matrix (arewep2pyet.com) exists is precisely to provide a 1st class P2P mode to Matrix - it's not just for awesome reliability and network resilience, but for easier onboarding; killing off public-service homeservers like matrix.org; etc. The onboarding and identity piece is covered by MSC4014 (which just got implemented in Dendrite.) However, you're totally right that our main focus is on the federated use case, because that's where everyone is today, and its own fair share of Hard Problems (e.g. reliable decentralised E2EE, byzantine-fault-tolerant conversation replication, decentralised ACLs, lazy-loaded conversation state) before we chuck P2P into the mix too. The P2P Matrix project is currently focusing mainly on MSC4014 (i.e. pseudo-IDs, account portability, multihomed accounts) which benefits both P2P and normal Matrix... although I'm hoping that we'll get back to Pinecone & actual P2P work eventually (especially if someone explicitly funds it). Meanwhile, it's super cool to see the progress on Quiet and OrbitDB. It's come a long way since Orbit launched at the 2016 Decentralised Web Summit, and I'm glad to see the tech is still progressing. FWIW, We did the first versions of P2P Matrix on top of libp2p (but then switched to Yggdrasil and then Pinecone in order to work with a monolithic stack, for better or worse). Might be interesting to figure out how the projects could interop in future :)
      - eTomte 3y ago

What about Veilid?

by tyzoid 3y ago

What about Veilid?

Veilid is a recently announced, general-purpose peer-to-peer Stack. We're excited about it! We would consider switching away from OrbitDB/IPFS/Libp2p/Tor for a less mature peer-to-peer state syncing stack if it offered the following as first-class features in addition to what we have, had credible longevity, and ran well on desktop and mobile: 1. Deletion 2. Encrypted groups with removal 3. Support for linking multiple devices, and removing devices 4. Support for partial syncing / lazy loading. It would be really nice to have all of these things out of the box, alongside state syncing and BitTorrent-style file transfer (which we have now.) Veilid is still new so we haven't assessed it yet, but that would be our criteria. My general perspective on general purpose peer-to-peer stacks (which I understand Veilid to be) is that it will be very difficult to build meaningful general purpose platforms until we have more examples of popular desktop + mobile peer-to-peer products with large numbers of users in the wild that would let us identify all the things a general purpose stack would need to have, and that right now the best thing to do is build single-purpose peer-to-peer apps in whatever way you can (whether it's by building your own non-general stack or cobbling together existing building blocks as we're doing) and struggle to find product market fit! The way I think about it is: would you have been able to build Heroku before there were a few successful production Ruby on Rails apps with product-market fit? Probably not! So we shouldn't expect to be able to build useful general purpose p2p stacks without clear examples of successful p2p products with product-market fit.

MayeulC 3y ago
Hmm, it feels like a CRDT + any overlay network (This is probably how you use tor or IPFS p2p) would get you there. Matrix p2p is developing pinecone [1] as an overlay network, which seems very aligned with your needs, though, as you say, it is less mature. Well, pinecone/yggdrasil/cjdns/tor are only part of the question, as they provide encrypted E2E connectivity. You still have to build on top of it to handle groups and lazy loading. They do solve routing though, so you can just try to send packets to every group member (like Matrix does). For a more efficient implementation, it would be nice to have multicast/unicast integrated in the overlay network at the protocol level. API-wise, it could be made as simple as sharing a private key with a group: multiple peers with the same address would then receive the same packets. In any case, I feel like working together with Matrix (/new vector) on pinecone would help both of you. [1]: https://github.com/matrix-org/pinecone https://github.com/matrix-org/pinecone
- holmesworcester 3y ago
  I've been following the P2P Matrix work intermittently but closely. Matrix clients are so much more mature than ours, so having a first-class P2P mode in Matrix would be a solution to this whole problem. That said, they have to solve more problems than just the problems solved by Pinecone. (Like onboarding and identity.) And over the past few years my impression has been that the Matrix team is focused on the federated use case, because that's where their users are. It seems hard for them to pay attention to both. My guess is that P2P Matrix will land as an awesome reliability feature for uninterrupted messaging when a server goes down or when the Internet goes out, but that it will take longer for them to tackle problems specific to P2P that don't graft on easily to federated Matrix.
  - Arathorn 3y ago
    The reason that P2P Matrix (arewep2pyet.com) exists is precisely to provide a 1st class P2P mode to Matrix - it's not just for awesome reliability and network resilience, but for easier onboarding; killing off public-service homeservers like matrix.org; etc. The onboarding and identity piece is covered by MSC4014 (which just got implemented in Dendrite.) However, you're totally right that our main focus is on the federated use case, because that's where everyone is today, and its own fair share of Hard Problems (e.g. reliable decentralised E2EE, byzantine-fault-tolerant conversation replication, decentralised ACLs, lazy-loaded conversation state) before we chuck P2P into the mix too. The P2P Matrix project is currently focusing mainly on MSC4014 (i.e. pseudo-IDs, account portability, multihomed accounts) which benefits both P2P and normal Matrix... although I'm hoping that we'll get back to Pinecone & actual P2P work eventually (especially if someone explicitly funds it). Meanwhile, it's super cool to see the progress on Quiet and OrbitDB. It's come a long way since Orbit launched at the 2016 Decentralised Web Summit, and I'm glad to see the tech is still progressing. FWIW, We did the first versions of P2P Matrix on top of libp2p (but then switched to Yggdrasil and then Pinecone in order to work with a monolithic stack, for better or worse). Might be interesting to figure out how the projects could interop in future :)
    - eTomte 3y ago