5 ms·

The problems with P2P is not problems of tech, they’re problems of incentives. The tech exists and it’s mostly solved — it’s just that anyone who is capable of

The problems with P2P is not problems of tech, they’re problems of incentives. The tech exists and it’s mostly solved — it’s just that anyone who is capable of building such a thing is not incentivised to do so, because that work could be spent in more productive (i.e. money-making) endeavours than what will by definition be incapable of making money since it explicitly decentralises the concentration of power. So if Jack wants you to come build it, and you’re a P2P person, you should go work for him in even if you don’t like him, because he has both the willingness to pay for it, and the broadcast power to make it mainstream. And those are the main problems that needs to be solved, not tech. I know this because I’m going through the same thing myself — I work on Aether (https://getaether.net https://getaether.net), which is close to what I think he is talking about. If we want ourselves to solve this, and not rely on @jacks of the world, we need to find a way to make P2P into a viable business. For me, in the end I decided to create a Pro version for use within companies, so that the P2P version can be fully free of monetisation concerns.

dmitriid 7y ago
P2P is not really solved. Not to the true satisfaction. For example, there are tens of millions of images uploaded to Twitter daily. They all have to be stored, and then retrieved for nearly arbitrary tweets and arbitrary users. All this content has to be stored somewhere and retrieved fast enough to be useable.
- dreamcompiler 7y ago
  Storing images in multiple, decentralized servers and making them available to arbitrary users instantly? Pretty sure that's a solved problem. </IMG>
  - josh2600 7y ago
    I'm pretty sure no major service (petabyte+ scale) functions this way.
    - buzzkillington 7y ago
      Because anyone who tried that got burned with child porn and the threat of real prison time. p2p tech is fundamentally incompatible with American morals. You will never see a p2p based firm be successful in the US any more than you could have had the industrial revolution in Ancien Regime France.
    - pcmonk 7y ago
      If it's actually p2p, you don't need petabyte scale. You just need to store what you post, which for most people is megabyte-to-gigabyte scale.
      - PeterisP 7y ago
        Each user storing what they post doesn't really scale. If I have a 100kb image in a post that gets popular, then the following technical things are needed: 1) It needs to work reasonably fast when the suddenly popular post is seen by, say, 10000 people in a short time. My home uplink won't handle that. 2) It needs to work when my computer is turned off or disconnected from the internet. 3) (optional, but still important) People on the other side of the world should be able to get popular data from a cached copy that's physically closer, both for latency and bandwidth reasons. "Store what you post" isn't sufficient, all that I post needs to be stored also by someone else (e.g. a CDN) - but that immediately rises all kinds of issues about liability. If only I am storing what I post, then the system can afford to not care if that image turns out to be child porn, but if someone else is storing what I post, then that's a problem for everyone.
  - rhizome 7y ago
    Jeez man, most sites have trouble putting static text on a page without showing a spinner.
    - dgellow 7y ago
      That’s for different reasons :)
      - rhizome 7y ago
        I'm not sure they're as different as you might think.
- lowkeyokay 7y ago
  PiedPiper solved this
  - gnaritas 7y ago
    lol, fuck the downvoters, that was funny.
  - sjg007 7y ago
    This comment is underrated. The theory is solid but piped piper doesn't exist in reality. Striping content across the network is the way to go. STORJ apparently is working on it as is the Interplanetary internet idea. It reminds me of FidoNet back in the day.... The internet we have exists because of low latency networks.. the question is how do we maintain that illusion? https://www.wired.com/2017/06/pied-pipers-new-internet-isnt-just-possible-almost/ https://www.wired.com/2017/06/pied-pipers-new-internet-isnt-...
buzzkillington 7y ago
Thanks for the great project, it fixes the most egregious problems with bulletin boards - which reddit and hn are just fancier guis for. There's a lot to unpack about the issues of p2p monetization. Probably a few books worth. Before we get into the social issues I think the original sin of the internet was that IP did not negotiate payments as part of the fundamental protocol. Every layer above that suffers from the same. This made sense in a government/corporate/university environment where the protocols were invented, but not for the wider world. I'm not going to try and give solutions because all of them are flawed in some way, but better than the current system where only surveillance advertising and altruistic services are viable business models.
- nradov 7y ago
  Payment is far too complex and abstract to contemplate building in at layer 3 (IP) of the network stack. Even in some idealized alternate history it would have never worked, and probably still couldn't work today.
  - buzzkillington 7y ago
    Global payment, yes. Same as global routing. The internet is happily chugging along on routers and switches who know nothing about the global topology of the network. When you reduce the payment problem to 'how do we get two routers to keep track of the data passed between them' it is a non-problem that even the first generation of imps could have solved).
    - warkdarrior 7y ago
      Except for the part where you do not want payments to be best effort, while it is OK for network packet delivery to be best effort.
      - buzzkillington 7y ago
        Why? I don't care that I will need to pay 1e-10 dollars to have a packet sent 10 times. Likewise I don't care if I lose 1e-10 dollars because someone doesn't pay for their bandwidth. Networking is just not worth lying about until you get to sums for which lawyers can get involved, at which point you have to deal with lawyers.
holler 7y ago
thanks for sharing, I just went through your docs trying to understand the architecture and realized that by nature your system has to erase data after a certain period, otherwise individual nodes would end up retaining potentially large amounts of data. Is that correct? Could a flood or gossip based system have full message history without ending up grinding to a slow halt?
- rolleiflex 7y ago
  Yes, this is in fact a privacy feature - the data is gone 6 months after its last reference. In other words, we garbage collect for you so that you won’t end up dealing with a post you wrote when you were 14 10 years later, out of context. It’s sort of like the right to be forgotten. That also makes it relatively unsuitable for long-term storage of stale information. Active communities will last, but archived ones with no new content won’t, and eventually be gone.
  - deleted 7y ago
    [deleted]
  - bilbo0s 7y ago
    Is there anything that stops an organization from just vacuuming up and archiving what it finds on these nodes? I mean, can some company come along and build "archived searchable aether" as a service?
    - rolleiflex 7y ago
      No. This is about not collecting data by default, so what you mention is certainly possible. That said, if you stop posting, your profile would also be gone in six months, so a third party would no longer have an entity to correlate the history to. Histories are useful only if they relate to an active entity and you have the option to kill that entity and move over to some other name - and that new name would be impossible to link from the old one.
      - 7y ago
michaelsbradley 7y ago
Incentivisation is a problem that's faced by many decentralized technologies. For example, one of the problems with Ethereum's Whisper protocol (decentralized messaging) has been that nodes aren't incentivised to process messages. There are other problems too and some efforts are being made to generally improve the tech[+]. [+] https://vac.dev/fixing-whisper-with-waku https://vac.dev/fixing-whisper-with-waku
rch 7y ago
Halfway through reading your comment I was going to mention Aether as a counterpoint. It's a great project that I'm happy to support even if I don't use it as often as I'd like.
- rolleiflex 7y ago
  Thank you! Your support is very much appreciated. (Context: I have a Patreon here: https://patreon.com/nehbit https://patreon.com/nehbit) I generally try to only charge it on the months I push a new release on, and otherwise keep it paused. I’ve found that’s a good way for me to not stress out when my ‘day job’ gets too hectic.
hanniabu 7y ago
Cryptocurrency/blockchain is a great to power incentives and pay for development. I currently an paid this way to work on a self-sufficient project that operates in this manner. (We don't rely on any fundraising funds as with the more common projects you're familiar with)

dmitriid 7y ago

P2P is not really solved. Not to the true satisfaction. For example, there are tens of millions of images uploaded to Twitter daily. They all have to be stored, and then retrieved for nearly arbitrary tweets and arbitrary users. All this content has to be stored somewhere and retrieved fast enough to be useable.

dreamcompiler 7y ago
Storing images in multiple, decentralized servers and making them available to arbitrary users instantly? Pretty sure that's a solved problem. </IMG>
- josh2600 7y ago
  I'm pretty sure no major service (petabyte+ scale) functions this way.
  - buzzkillington 7y ago
    Because anyone who tried that got burned with child porn and the threat of real prison time. p2p tech is fundamentally incompatible with American morals. You will never see a p2p based firm be successful in the US any more than you could have had the industrial revolution in Ancien Regime France.
  - pcmonk 7y ago
    If it's actually p2p, you don't need petabyte scale. You just need to store what you post, which for most people is megabyte-to-gigabyte scale.
    - PeterisP 7y ago
      Each user storing what they post doesn't really scale. If I have a 100kb image in a post that gets popular, then the following technical things are needed: 1) It needs to work reasonably fast when the suddenly popular post is seen by, say, 10000 people in a short time. My home uplink won't handle that. 2) It needs to work when my computer is turned off or disconnected from the internet. 3) (optional, but still important) People on the other side of the world should be able to get popular data from a cached copy that's physically closer, both for latency and bandwidth reasons. "Store what you post" isn't sufficient, all that I post needs to be stored also by someone else (e.g. a CDN) - but that immediately rises all kinds of issues about liability. If only I am storing what I post, then the system can afford to not care if that image turns out to be child porn, but if someone else is storing what I post, then that's a problem for everyone.
- rhizome 7y ago
  Jeez man, most sites have trouble putting static text on a page without showing a spinner.
  - dgellow 7y ago
    That’s for different reasons :)
    - rhizome 7y ago
      I'm not sure they're as different as you might think.
lowkeyokay 7y ago
PiedPiper solved this
- gnaritas 7y ago
  lol, fuck the downvoters, that was funny.
- sjg007 7y ago
  This comment is underrated. The theory is solid but piped piper doesn't exist in reality. Striping content across the network is the way to go. STORJ apparently is working on it as is the Interplanetary internet idea. It reminds me of FidoNet back in the day.... The internet we have exists because of low latency networks.. the question is how do we maintain that illusion? https://www.wired.com/2017/06/pied-pipers-new-internet-isnt-just-possible-almost/ https://www.wired.com/2017/06/pied-pipers-new-internet-isnt-...

buzzkillington 7y ago

Thanks for the great project, it fixes the most egregious problems with bulletin boards - which reddit and hn are just fancier guis for. There's a lot to unpack about the issues of p2p monetization. Probably a few books worth. Before we get into the social issues I think the original sin of the internet was that IP did not negotiate payments as part of the fundamental protocol. Every layer above that suffers from the same. This made sense in a government/corporate/university environment where the protocols were invented, but not for the wider world. I'm not going to try and give solutions because all of them are flawed in some way, but better than the current system where only surveillance advertising and altruistic services are viable business models.

nradov 7y ago
Payment is far too complex and abstract to contemplate building in at layer 3 (IP) of the network stack. Even in some idealized alternate history it would have never worked, and probably still couldn't work today.
- buzzkillington 7y ago
  Global payment, yes. Same as global routing. The internet is happily chugging along on routers and switches who know nothing about the global topology of the network. When you reduce the payment problem to 'how do we get two routers to keep track of the data passed between them' it is a non-problem that even the first generation of imps could have solved).
  - warkdarrior 7y ago
    Except for the part where you do not want payments to be best effort, while it is OK for network packet delivery to be best effort.
    - buzzkillington 7y ago
      Why? I don't care that I will need to pay 1e-10 dollars to have a packet sent 10 times. Likewise I don't care if I lose 1e-10 dollars because someone doesn't pay for their bandwidth. Networking is just not worth lying about until you get to sums for which lawyers can get involved, at which point you have to deal with lawyers.

holler 7y ago

thanks for sharing, I just went through your docs trying to understand the architecture and realized that by nature your system has to erase data after a certain period, otherwise individual nodes would end up retaining potentially large amounts of data. Is that correct? Could a flood or gossip based system have full message history without ending up grinding to a slow halt?

rolleiflex 7y ago
Yes, this is in fact a privacy feature - the data is gone 6 months after its last reference. In other words, we garbage collect for you so that you won’t end up dealing with a post you wrote when you were 14 10 years later, out of context. It’s sort of like the right to be forgotten. That also makes it relatively unsuitable for long-term storage of stale information. Active communities will last, but archived ones with no new content won’t, and eventually be gone.
- deleted 7y ago
  [deleted]
- bilbo0s 7y ago
  Is there anything that stops an organization from just vacuuming up and archiving what it finds on these nodes? I mean, can some company come along and build "archived searchable aether" as a service?
  - rolleiflex 7y ago
    No. This is about not collecting data by default, so what you mention is certainly possible. That said, if you stop posting, your profile would also be gone in six months, so a third party would no longer have an entity to correlate the history to. Histories are useful only if they relate to an active entity and you have the option to kill that entity and move over to some other name - and that new name would be impossible to link from the old one.
    - 7y ago

michaelsbradley 7y ago

Incentivisation is a problem that's faced by many decentralized technologies. For example, one of the problems with Ethereum's Whisper protocol (decentralized messaging) has been that nodes aren't incentivised to process messages. There are other problems too and some efforts are being made to generally improve the tech[+]. [+] https://vac.dev/fixing-whisper-with-waku https://vac.dev/fixing-whisper-with-waku

rch 7y ago

Halfway through reading your comment I was going to mention Aether as a counterpoint. It's a great project that I'm happy to support even if I don't use it as often as I'd like.

rolleiflex 7y ago
Thank you! Your support is very much appreciated. (Context: I have a Patreon here: https://patreon.com/nehbit https://patreon.com/nehbit) I generally try to only charge it on the months I push a new release on, and otherwise keep it paused. I’ve found that’s a good way for me to not stress out when my ‘day job’ gets too hectic.

hanniabu 7y ago

Cryptocurrency/blockchain is a great to power incentives and pay for development. I currently an paid this way to work on a self-sufficient project that operates in this manner. (We don't rely on any fundraising funds as with the more common projects you're familiar with)