7 ms·

(Quiet founder here) All of these things are true and it's clear you know the problem space well! We avoid the primary "go brrrrr" performance issue with IPFS

(Quiet founder here) All of these things are true and it's clear you know the problem space well! We avoid the primary "go brrrrr" performance issue with IPFS by using small private IPFS networks and never touching the noisy, CPU-intensive global network. You didn't even mention the biggest time drain, which is getting everything to work well on Android and iOS! This is on track to dwarf all other time drains by 10x or more. However, the amazing thing is, as full-of-warts as the stack we've chosen is, it isn't a clearly worse choice than the alternatives, from GUN, to Hypercore, to the Automerge space, to Braid, to Holochain, P2Ppanda, and so on. All the other attempts to build a general purpose stack have a few things they do well and a bunch of things they don't do well, or at all, or they have similar questionable longevity to something like OrbitDB. But if you believe what we both believe (i.e. that users shouldn't have to depend on someone else's server or run their own just to gather online) what is to be done? It seems like you decided to roll up your sleeves and try. Maybe the necessary ingredient is PG's "Schlep Blindness" [1] where you have to be pretty naive or masochistic to get started but if you keep pushing you can make something really new and valuable. I think the reason why the field is in this place is simply that a general purpose p2p stack for user-facing apps needs to do the impossible: it needs to build complex solutions to an actually very long list of heterogeneous problems, without clear knowledge of what those problems are and what form they take in production, because there aren't any successful, desktop + mobile peer-to-peer apps yet to provide that reality-tested information, unless you count WebRTC video calls, which only covers a small piece of the picture. So we just have to push through. And it's fun! Also, please do DM or email me! I'd love to compare notes more deeply than would be possible here (same offer goes for others who are interested in these questions!): h [at] quiet.chat 1. http://www.paulgraham.com/schlep.html http://www.paulgraham.com/schlep.html

mrusme 3y ago
I appreciate the reply and I understand where you're coming from. Looking at when you released your first few versions of this project, I understand that the options you had at that time probably weren't as plentyful or mature as they might be today. > However, the amazing thing is, as full-of-warts as the stack we've chosen is, it isn't a clearly worse choice than the alternatives, from GUN, to Hypercore, to the Automerge space, to Braid, to Holochain, P2Ppanda, and so on. I think far more interesting these days would be projects like Veilid, Hyphanet's Locutus and ultimately Nostr -- even though not truly P2P in that sense -- which already happens to have a first try going with nostrchat.io. If P2P is something that is truly desired, I feel like projects like Briar (https://briarproject.org/how-it-works/ https://briarproject.org/how-it-works/) have solved this with Bramble (https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BTP.md https://code.briarproject.org/briar/briar-spec/blob/master/p...) more eloquently than it could be done on top of IPFS. > It seems like you decided to roll up your sleeves and try. Ultimately IPFS was designed as a file store, which to me was the main reason to use it for Superhighway84. I intended to implement USENET-like file attachments and (go-)OrbitDB was merely an easier way to deal with the messages posted into the groups. However, if instant communication would be my main focus, I would rather try more lightweight frameworks (see above), especially if mobile devices should come into play eventually. Superhighway84 was never intended to leave the beautiful space that is the command line and hence niche-by-design. :-) Again, I will definitely keep an eye on Quiet, to see how it ultimately evolves on top of IPFS. I could nevertheless imagine it being overtaken fairly quickly by other projects sporting a rather lightweight and more managable basis, that allows for increased development speed and ultimately for faster iteration on features that users might wish for (e.g. DMs, @-mentions, message deletion, mobile clients, you-name-it) -- without the need to invest heavily into e.g. performance (or reliability!) issues of the underlying framework.
- holmesworcester 3y ago
  > I think far more interesting these days would be projects like Veilid, Hyphanet's Locutus I have not assessed Veilid yet but it's on my list and at a first glance seems like a very serious and informed attempt. I'm personal friends with Freenet / Hyphanet's Ian Clarke and spoke with him about Locutus when he was just getting started. It sounded awesome then and I will give this a second look too, though when he explained it to me it sounded like it had the same limitations with deletion that Nostr or the global IPFS network would have. It does seem important to note here that both Veilid and Locutus are much less mature and battle-tested than libp2p and Tor and have less Lindy longevity (longevity as a function of age.) We already suffer a lot from being on the bleeding edge, so it's nice to limit the number of bleeding edge tools we use. Libp2p, notably, has been rock solid for us and barely a time drain at all, apart from some unexpected interactions with Tor which are mostly about the lack of an official first-class Tor transport, which is specific to our use case and should start to change soon when Tor's Arti is ready. > and ultimately Nostr -- even though not truly P2P in that sense -- which already happens to have a first try going with nostrchat.io. Nostr and Bluesky both seem very promising for the open-world use case of social networking, and it has been amazing to see Nostr grow so rapidly as a community. I am rooting for this project and we might use it someday in Quiet for public feeds. Timed deletion is the user requirement that drives me away from building Quiet on Nostr. Based on conversations I've had with users doing sensitive work (and based on my own experience as a founder of Fight for the Future) timed deletion is extremely important to team security, and for deletion to be meaningful one needs more control over where the data is relayed than what Nostr provides in the default mode. A group that wanted trustworthy timed deletion would have to control their own private Nostr relay. Technically, a Tor relay could subvert the timed deletion of some Quiet messages just by capturing all traffic, but this is much less of a worry. > If P2P is something that is truly desired, I feel like projects like Briar (https://briarproject.org/how-it-works/ https://briarproject.org/how-it-works/) have solved this with Bramble (https://code.briarproject.org/briar/briar-spec/blob/master/p https://code.briarproject.org/briar/briar-spec/blob/master/p...) more eloquently than it could be done on top of IPFS. Bramble could work for us and I would recommend that anyone look into it. Briar is probably the most similar thing to Quiet that exists right now. There are big differences between Quiet and Briar, but we could definitely build Quiet on Bramble if it adequately supports iOS. My worry would be its maturity as a tool for people building things other than Briar. That could be worth the risk though and I do recommend anyone else reading this thread look at Bramble if you are doing something similar. > I could nevertheless imagine it being overtaken fairly quickly by other projects sporting a rather lightweight and more managable basis, that allows for increased development speed and ultimately for faster iteration on features that users might wish for (e.g. DMs, @-mentions, message deletion, mobile clients, you-name-it) -- without the need to invest heavily into e.g. performance (or reliability!) issues of the underlying framework. This is definitely something we will keep an eye on, and thank you for the thoughtful advice! My guess is that as soon as we have a significant number of real users we will need to build things that don't happen to be supported by whatever stack we choose (whether that is our current stack, Bramble, Veilid, Automerge, etc.) So the question is what's the easiest one to maintain and adapt. So far libp2p and IPFS have both been good in that department: implementations in many languages, active development, an absence of major problems showing signs of maturity (especially in libp2p), etc. Also, my 2 cents are (for anyone following along) that if I had to do this all over again I would use Tor + Libp2p + Automerge. Libp2p and Gossipsub are solid, flexible, and will be around a while. No need to reinvent the wheel. The conceptual framework behind Automerge and Briar/Bramble are pretty similar (sync state!) but the Automerge team exists to serve people building other apps, while the Bramble team mostly focuses on Briar AFAIK. What's nice about Automerge is that the community around it (Ink & Switch, Martin Kleppmann, and other academics) is all at the academic frontier, so the level of thought and anticipation of user needs that goes into their decisions is very thorough, even if the implementations lag behind the papers. If I was doing real-time p2p text editing I would also look at the Braid project (braid.org) and Seph Gentle's work on Diamond Types, since that's where the most thought has gone into the raw performance you need for text CRDTs that can handle large documents: https://github.com/josephg/diamond-types https://github.com/josephg/diamond-types
horsawlarway 3y ago
> i.e. that users shouldn't have to depend on someone else's server or run their own just to gather online I don't understand this. Can you elaborate exactly what you mean? Because to me... you're now just depending on a whole bunch of other people's machines indirectly, and directly on the community owner's machine which is generating the certs. It feels like a lot of complexity for something that could just be a small chat server running on the community owner's server (which they will need anyways - unless I'm misunderstanding, which is entirely possible). --- So since I'm probably missing something - can I get the elevator pitch? Assume I'm your target market (I want private messaging that I control). I would likely be a "community owner" as described in your article. I am already running a self-hosted solution (ex: Zulip/Rocket/Mattermost). What makes this a compelling offering to me?
- gabereiser 3y ago
  You aren’t missing anything. The restriction to community fences ensures that each community will have to host the community. There’s no free lunch. Now, someone in that community can be more generous with compute than others. Using Tor to try to be anonymous isn’t going to work either as Tor has been broken.
  - Renaud 3y ago
    > Tor has been broken I think you have to clarify what you mean by that. Citation needed.
    - reactordev 3y ago
      Sure thing. Let me know if you need more. Government agencies have been watching for years. Also keep in mind that no one has more admin access to network infrastructure than government agencies do such that the NSA can monitor any computer on the internet. https://arstechnica.com/tech-policy/2015/01/did-feds-mount-a-sustained-attack-on-tor-to-decloak-crime-suspects/ https://arstechnica.com/tech-policy/2015/01/did-feds-mount-a... https://www.documentcloud.org/documents/2719591-Farrell-Weds.html https://www.documentcloud.org/documents/2719591-Farrell-Weds... https://mice.cs.columbia.edu/getTechreport.php?techreportID=1545&format=pdf https://mice.cs.columbia.edu/getTechreport.php?techreportID=... https://www.vice.com/en/article/4x3qnj/how-the-nsa-or-anyone-else-can-crack-tors-anonymity https://www.vice.com/en/article/4x3qnj/how-the-nsa-or-anyone... https://www.techtimes.com/articles/200592/20170307/fbi-drops-child-pornography-case-to-avoid-disclosing-tor-vulnerability.htm https://www.techtimes.com/articles/200592/20170307/fbi-drops...
motohagiography 3y ago
The hard part about making a new P2P protocol is that first you have to invent the internet.
- holmesworcester 3y ago
  this statement works on a number of levels!
  - motohagiography 3y ago
    About 7 I'd reckon.
panick21_ 3y ago
Have you seen the work Matrix and Element have done on p2p. https://arewep2pyet.com/ https://arewep2pyet.com/ I have high hopes for this, its slow in development but its a difficult problem. If they can get it to work it can reuse the whole rest of the ecosystem.
rcxdude 3y ago
> because there aren't any successful, desktop + mobile peer-to-peer apps yet to provide that reality-tested information, unless you count WebRTC video calls, which only covers a small piece of the picture Not that it really managed much in mobile, but old-school Skype comes to mind as the most widely used P2P messaging application. That's probably where a lot of the most valuable 'at scale' knowledge is/was.
- holmesworcester 3y ago
  yes! i'm curious how much of what they did would be possible on mobile.
salawat 3y ago
Out of curiousity, what os your plan w.r.t the business? Given that Skype as it was originally implemented was very nearly this (P2P comms), and was targeted specifically for acquisition by Microsoft by pressure from intelligence agencies (to be re-implememted in a centralized fashion for tappability, see PRISM); I try to encourage every eager startup founder to think about their personal exit early. Any type of software offering that is done as a commercial venture lasts only as long as that founder/idealist is at the helm and there remains enough technically savvy people to fork on the inevitable rugpull. Which from your tech stack, may be an issue. Anything like this, while noble, is going to inevitably become a hot target for law enforcement/intelligence agency/nation state compromise, or media smear campaign the first time a bad actor comes to light who has been enabled by it. Prepare for this type of stuff as early as possible, and godspeed. Also, how'd you tackle the key distribution nut? Which is the hardest part of the entire process, in my experience. PKI?
- holmesworcester 3y ago
  Great questions and advice! Re: business plans, ideally we'll sell premium subscriptions for features you need a server for, like video calls. The biggest difference between us and Skype is that Quiet is open source. But yes, open source businesses can rugpull too, as we saw recently with Terraform. What about our stack makes you worried about the "enough tech savvy people to fork" piece? One decision we've made deliberately is to build on the most widely-used tech, so that maintenance will require less expertise than for a homegrown stack, and so there will be existing communities around the stack that are bigger than the Quiet community. I would love to know more about what problem you envision in building a tech-savvy open source community around our stack. Too boring? If our business is upselling users to server-backed subscription plans, I think even the threat of a fork goes a long way to keeping us honest, especially since a community fork would not need to run infrastructure. If "Quiet Co." (or whatever we call ourselves) is suddenly no longer the most trusted purveyor of Quiet, we wouldn't have much of a business, which is as it should be in my view. Re: the politics of providing these tools, I have been preparing, and I have some background in the political side of this from Fight for the Future. It's funny because I am actually quite eager to get to the point where we get to make the social and political case for Quiet to a partly-skeptical world, but first I have to make something that works well on phones! And find users! Ideally we can find some awesome initial users that really tell the story of why Quiet needs to exist.

mrusme 3y ago

I appreciate the reply and I understand where you're coming from. Looking at when you released your first few versions of this project, I understand that the options you had at that time probably weren't as plentyful or mature as they might be today. > However, the amazing thing is, as full-of-warts as the stack we've chosen is, it isn't a clearly worse choice than the alternatives, from GUN, to Hypercore, to the Automerge space, to Braid, to Holochain, P2Ppanda, and so on. I think far more interesting these days would be projects like Veilid, Hyphanet's Locutus and ultimately Nostr -- even though not truly P2P in that sense -- which already happens to have a first try going with nostrchat.io. If P2P is something that is truly desired, I feel like projects like Briar (https://briarproject.org/how-it-works/ https://briarproject.org/how-it-works/) have solved this with Bramble (https://code.briarproject.org/briar/briar-spec/blob/master/protocols/BTP.md https://code.briarproject.org/briar/briar-spec/blob/master/p...) more eloquently than it could be done on top of IPFS. > It seems like you decided to roll up your sleeves and try. Ultimately IPFS was designed as a file store, which to me was the main reason to use it for Superhighway84. I intended to implement USENET-like file attachments and (go-)OrbitDB was merely an easier way to deal with the messages posted into the groups. However, if instant communication would be my main focus, I would rather try more lightweight frameworks (see above), especially if mobile devices should come into play eventually. Superhighway84 was never intended to leave the beautiful space that is the command line and hence niche-by-design. :-) Again, I will definitely keep an eye on Quiet, to see how it ultimately evolves on top of IPFS. I could nevertheless imagine it being overtaken fairly quickly by other projects sporting a rather lightweight and more managable basis, that allows for increased development speed and ultimately for faster iteration on features that users might wish for (e.g. DMs, @-mentions, message deletion, mobile clients, you-name-it) -- without the need to invest heavily into e.g. performance (or reliability!) issues of the underlying framework.

holmesworcester 3y ago
> I think far more interesting these days would be projects like Veilid, Hyphanet's Locutus I have not assessed Veilid yet but it's on my list and at a first glance seems like a very serious and informed attempt. I'm personal friends with Freenet / Hyphanet's Ian Clarke and spoke with him about Locutus when he was just getting started. It sounded awesome then and I will give this a second look too, though when he explained it to me it sounded like it had the same limitations with deletion that Nostr or the global IPFS network would have. It does seem important to note here that both Veilid and Locutus are much less mature and battle-tested than libp2p and Tor and have less Lindy longevity (longevity as a function of age.) We already suffer a lot from being on the bleeding edge, so it's nice to limit the number of bleeding edge tools we use. Libp2p, notably, has been rock solid for us and barely a time drain at all, apart from some unexpected interactions with Tor which are mostly about the lack of an official first-class Tor transport, which is specific to our use case and should start to change soon when Tor's Arti is ready. > and ultimately Nostr -- even though not truly P2P in that sense -- which already happens to have a first try going with nostrchat.io. Nostr and Bluesky both seem very promising for the open-world use case of social networking, and it has been amazing to see Nostr grow so rapidly as a community. I am rooting for this project and we might use it someday in Quiet for public feeds. Timed deletion is the user requirement that drives me away from building Quiet on Nostr. Based on conversations I've had with users doing sensitive work (and based on my own experience as a founder of Fight for the Future) timed deletion is extremely important to team security, and for deletion to be meaningful one needs more control over where the data is relayed than what Nostr provides in the default mode. A group that wanted trustworthy timed deletion would have to control their own private Nostr relay. Technically, a Tor relay could subvert the timed deletion of some Quiet messages just by capturing all traffic, but this is much less of a worry. > If P2P is something that is truly desired, I feel like projects like Briar (https://briarproject.org/how-it-works/ https://briarproject.org/how-it-works/) have solved this with Bramble (https://code.briarproject.org/briar/briar-spec/blob/master/p https://code.briarproject.org/briar/briar-spec/blob/master/p...) more eloquently than it could be done on top of IPFS. Bramble could work for us and I would recommend that anyone look into it. Briar is probably the most similar thing to Quiet that exists right now. There are big differences between Quiet and Briar, but we could definitely build Quiet on Bramble if it adequately supports iOS. My worry would be its maturity as a tool for people building things other than Briar. That could be worth the risk though and I do recommend anyone else reading this thread look at Bramble if you are doing something similar. > I could nevertheless imagine it being overtaken fairly quickly by other projects sporting a rather lightweight and more managable basis, that allows for increased development speed and ultimately for faster iteration on features that users might wish for (e.g. DMs, @-mentions, message deletion, mobile clients, you-name-it) -- without the need to invest heavily into e.g. performance (or reliability!) issues of the underlying framework. This is definitely something we will keep an eye on, and thank you for the thoughtful advice! My guess is that as soon as we have a significant number of real users we will need to build things that don't happen to be supported by whatever stack we choose (whether that is our current stack, Bramble, Veilid, Automerge, etc.) So the question is what's the easiest one to maintain and adapt. So far libp2p and IPFS have both been good in that department: implementations in many languages, active development, an absence of major problems showing signs of maturity (especially in libp2p), etc. Also, my 2 cents are (for anyone following along) that if I had to do this all over again I would use Tor + Libp2p + Automerge. Libp2p and Gossipsub are solid, flexible, and will be around a while. No need to reinvent the wheel. The conceptual framework behind Automerge and Briar/Bramble are pretty similar (sync state!) but the Automerge team exists to serve people building other apps, while the Bramble team mostly focuses on Briar AFAIK. What's nice about Automerge is that the community around it (Ink & Switch, Martin Kleppmann, and other academics) is all at the academic frontier, so the level of thought and anticipation of user needs that goes into their decisions is very thorough, even if the implementations lag behind the papers. If I was doing real-time p2p text editing I would also look at the Braid project (braid.org) and Seph Gentle's work on Diamond Types, since that's where the most thought has gone into the raw performance you need for text CRDTs that can handle large documents: https://github.com/josephg/diamond-types https://github.com/josephg/diamond-types

horsawlarway 3y ago

> i.e. that users shouldn't have to depend on someone else's server or run their own just to gather online I don't understand this. Can you elaborate exactly what you mean? Because to me... you're now just depending on a whole bunch of other people's machines indirectly, and directly on the community owner's machine which is generating the certs. It feels like a lot of complexity for something that could just be a small chat server running on the community owner's server (which they will need anyways - unless I'm misunderstanding, which is entirely possible). --- So since I'm probably missing something - can I get the elevator pitch? Assume I'm your target market (I want private messaging that I control). I would likely be a "community owner" as described in your article. I am already running a self-hosted solution (ex: Zulip/Rocket/Mattermost). What makes this a compelling offering to me?

gabereiser 3y ago
You aren’t missing anything. The restriction to community fences ensures that each community will have to host the community. There’s no free lunch. Now, someone in that community can be more generous with compute than others. Using Tor to try to be anonymous isn’t going to work either as Tor has been broken.
- Renaud 3y ago
  > Tor has been broken I think you have to clarify what you mean by that. Citation needed.
  - reactordev 3y ago
    Sure thing. Let me know if you need more. Government agencies have been watching for years. Also keep in mind that no one has more admin access to network infrastructure than government agencies do such that the NSA can monitor any computer on the internet. https://arstechnica.com/tech-policy/2015/01/did-feds-mount-a-sustained-attack-on-tor-to-decloak-crime-suspects/ https://arstechnica.com/tech-policy/2015/01/did-feds-mount-a... https://www.documentcloud.org/documents/2719591-Farrell-Weds.html https://www.documentcloud.org/documents/2719591-Farrell-Weds... https://mice.cs.columbia.edu/getTechreport.php?techreportID=1545&format=pdf https://mice.cs.columbia.edu/getTechreport.php?techreportID=... https://www.vice.com/en/article/4x3qnj/how-the-nsa-or-anyone-else-can-crack-tors-anonymity https://www.vice.com/en/article/4x3qnj/how-the-nsa-or-anyone... https://www.techtimes.com/articles/200592/20170307/fbi-drops-child-pornography-case-to-avoid-disclosing-tor-vulnerability.htm https://www.techtimes.com/articles/200592/20170307/fbi-drops...

motohagiography 3y ago

The hard part about making a new P2P protocol is that first you have to invent the internet.

holmesworcester 3y ago
this statement works on a number of levels!
- motohagiography 3y ago
  About 7 I'd reckon.

panick21_ 3y ago

Have you seen the work Matrix and Element have done on p2p. https://arewep2pyet.com/ https://arewep2pyet.com/ I have high hopes for this, its slow in development but its a difficult problem. If they can get it to work it can reuse the whole rest of the ecosystem.

rcxdude 3y ago

> because there aren't any successful, desktop + mobile peer-to-peer apps yet to provide that reality-tested information, unless you count WebRTC video calls, which only covers a small piece of the picture Not that it really managed much in mobile, but old-school Skype comes to mind as the most widely used P2P messaging application. That's probably where a lot of the most valuable 'at scale' knowledge is/was.

holmesworcester 3y ago
yes! i'm curious how much of what they did would be possible on mobile.

salawat 3y ago

Out of curiousity, what os your plan w.r.t the business? Given that Skype as it was originally implemented was very nearly this (P2P comms), and was targeted specifically for acquisition by Microsoft by pressure from intelligence agencies (to be re-implememted in a centralized fashion for tappability, see PRISM); I try to encourage every eager startup founder to think about their personal exit early. Any type of software offering that is done as a commercial venture lasts only as long as that founder/idealist is at the helm and there remains enough technically savvy people to fork on the inevitable rugpull. Which from your tech stack, may be an issue. Anything like this, while noble, is going to inevitably become a hot target for law enforcement/intelligence agency/nation state compromise, or media smear campaign the first time a bad actor comes to light who has been enabled by it. Prepare for this type of stuff as early as possible, and godspeed. Also, how'd you tackle the key distribution nut? Which is the hardest part of the entire process, in my experience. PKI?

holmesworcester 3y ago
Great questions and advice! Re: business plans, ideally we'll sell premium subscriptions for features you need a server for, like video calls. The biggest difference between us and Skype is that Quiet is open source. But yes, open source businesses can rugpull too, as we saw recently with Terraform. What about our stack makes you worried about the "enough tech savvy people to fork" piece? One decision we've made deliberately is to build on the most widely-used tech, so that maintenance will require less expertise than for a homegrown stack, and so there will be existing communities around the stack that are bigger than the Quiet community. I would love to know more about what problem you envision in building a tech-savvy open source community around our stack. Too boring? If our business is upselling users to server-backed subscription plans, I think even the threat of a fork goes a long way to keeping us honest, especially since a community fork would not need to run infrastructure. If "Quiet Co." (or whatever we call ourselves) is suddenly no longer the most trusted purveyor of Quiet, we wouldn't have much of a business, which is as it should be in my view. Re: the politics of providing these tools, I have been preparing, and I have some background in the political side of this from Fight for the Future. It's funny because I am actually quite eager to get to the point where we get to make the social and political case for Quiet to a partly-skeptical world, but first I have to make something that works well on phones! And find users! Ideally we can find some awesome initial users that really tell the story of why Quiet needs to exist.