4 ms·

> Anyway, I came to the same conclusion as the author, but several years ago: in the end, nothing is actually decentralized, and maintaining this illusion of de

> Anyway, I came to the same conclusion as the author, but several years ago: in the end, nothing is actually decentralized, and maintaining this illusion of decentralization is actually costly, for no real purpose (other than the initial enjoyment of playing with a new tech, that is). Do you have any writing (blog posts, HN comments, etc.) where you explore this thought more? I'm in the thick of building p2p software, very interested in what you came to know during that time.

deleted 3y ago
[deleted]
pphysch 3y ago
True P2P networks don't scale, because every node has to store an (accurate if partial) representation of the whole network. Growing the network is easy, but growing every single node is virtually impossible (unless you control them all...). Any attempt to tackle that tends to increase centralization, e.g. in the form of routing authorities. And if you try to tackle centralization directly (like banning routers or something), you will often create an anti-centralization regulator, which is, you guessed it, another form of centralization. So your decentralized P2P network is either small and works good, medium and works not so good, or large and not actually decentralized. The best P2P networks know their limits and don't try to scale infinitely. For human-oriented network, Dunbar's Number (N=~150) is a decent rule of thumb; any P2P network larger than that almost certainly has some form of centralization (like trusted bootstrapping/coordination server addresses that are hard-coded in every client install, etc.)
- KMag 3y ago
  > True P2P networks don't scale, because every node has to store an (accurate if partial) representation of the whole network Former LimeWire dev here... which P2P networks use a fully meshed topology? LimeWire and other Gnutella clients just have a random mesh with a fixed number of (ultra)peers. If the network gets too large, then your constrained broadcast queries hit their hop count before reaching the edge of the network, but that seems fine. Last I checked, Freenet used a variation on a random mesh. Kademlia's routing tables take O(log(N)) space and traffic per-peer to maintain (so O(N log(N)) for global total network space and traffic). Same for Chord (though, twice as much traffic due to not using a symmetric distance metric like XOR). There are plenty of "True" (non-centralized) P2P networks that aren't fully meshed.
  - sanity 3y ago
    Creator of Freenet here. Freenet[1] relies on peers self-organizing into a small world network[2]. Small world networks have the advantage of being able to find data in log N time where N is the network size, they're also completely decentralized, self-healing, and distribute load evenly across peers. The principle is similar to DHTs like Kademlia but more flexible and intuitive IMO, while having similar scaling characteristics. It's surprisingly common for people to confuse small world networks with "scale free networks", but scale free networks rely on a subset of highly connected peers which do a disproportionate amount of the work - which isn't truly decentralized. The new Freenet design incorporates adaptive learning into the routing algorithm. When a peer is deciding where to route a message, it predicts the response probability and time for each neighboring peer based on past performance and chooses the best. With conventional "greedy routing", peers choose the neighbor with a location closest to the data being retrieved. The new approach is adaptive to actual network performance. [1] Both the original Freenet from 1999 and the new sequel we're currently building - see https://freenet.org/ https://freenet.org/ for more. We hope to launch the network in the next few weeks. [2] https://en.wikipedia.org/wiki/Small-world_network https://en.wikipedia.org/wiki/Small-world_network
    - KMag 3y ago
      Thanks for the great work, Ian! As far a second-generation Freenet, I heard i2p started out as a proposed re-factoring and generalization of Freenet's encrypted transport layer. Are there any plans on using i2p to carry Freenet traffic?
      - sanity 3y ago
        Thank you :) I2P was created by someone who was previously involved with Freenet, but its design is a lot closer to Tor than to Freenet. Both I2P and Tor are anonymizing proxies, they allow services to be hidden, but they're still centralized. While they are quite different, there is enough overlap that running Freenet over I2P (or Tor) would be wildly inefficient and slow, so I wouldn't recommend it. Freenet is designed to run over UDP directly. The new Freenet is designed to allow the creation of completely decentralized services. Briefly, it's a global key-value store in which keys are webassembly code that specify what values are permitted under that key, and the conditions under which those values can be modified. This key-value store is observable, so anyone can subscribe to a key and be notified immediately if the value changes. This is just scratching the surface, for anyone interested in a much more comprehensive explanation of the new Freenet please see this talk I gave a few months ago: [1] You can also find a FAQ here: [2] [1] https://www.youtube.com/watch?v=yBtyNIqZios https://www.youtube.com/watch?v=yBtyNIqZios [2] https://freenet.org/faq https://freenet.org/faq
p4bl0 3y ago
The main thing is that "true" (in the sense of absolute) decentralization does not actually work. It doesn't work technically, and it doesn't work socially/politically. We always need some form of collective control over what's going on. We need moderation tools, we need to me able to make errors and fix them, etc. Centralized systems tend to be authoritarians, but the pursue of absolute decentralization always end up being very individualistic (and some kind of wrongly placed elitism). There are middle grounds: federated systems for example, like Mastodon or emails, actually work. That is not to say that all p2p software is bad, especially since we call p2p a lot of things that are not entirely p2p. For example, BitTorrent is a p2p software, but its actual usage by humans relies on multiple more-or-less centralized point, trackers and torrent search engine.
- r3trohack3r 3y ago
  I think you and I would have a very interesting conversation. I agree with a lot of your points, but not your conclusions. > We always need some form of collective control over what's going on. We need moderation tools I agree with this, and also think it is possible in peer-to-peer systems. Ideally the collective is self-governed. Particularly, when it comes to moderation, the closer the moderation controls are to being under the control of the user consuming the content, the more just the system. > , we need to me able to make errors and fix them, etc. Yes, 100%. Equity < Cryptography. It's far more important that your equity be rock solid than it is for your cryptography to be rock solid. If someone steals your property in a cryptographically sound way, equity should always trump cryptography. It's far more important that you have the title to your vehicle than it is that you have the keys to your vehicle. I feel like many p2p systems have gotten this one backwards. > There are middle grounds: federated systems for example, like Mastodon or emails, actually work. I consider these centralized systems, just N copies of the same problem. I don't feel like the power imbalance between the users and the system administrators are addressed in a sufficient way on the fediverse as it is currently implemented. These systems have a classist, hierarchical, system where system administrators belong to a privileged class while users are second class citizens on the web. --- I feel one of the issues in the current centralized architectures is equity. When you go about moving through the world, you generate a large volume of valuable property (your data). But, today, you give away nearly all equity in that data. Centralized providers accumulate equity in their user's data, and that equity is how they pay the bills. I do believe that, in a very meaningful way, equity and privacy are nearly synonymous when it comes to corporations respecting the privacy of their users. Just because Netflix delivers a video to your smartphone doesn't mean you can turn around and sell that video to your friend. You have been granted access to the video, but it is not your property. The inverse needs to be true too. Just because you share your viewing habits with Netflix doesn't mean they can sell that data to Warner Brothers, that's not their property (I mean, today it is, but it shouldn't be). If users had equity in their data, the data broker market as it exists today would be piracy. P2P systems have failed to create a world where humans, their content, and their devices are meaningfully addressable for the web in a way that expresses equity as a first class citizen. A decentralized world is possible, it's just not possible on the internet. The internet, as it exists, is insufficient for expressing the concepts of the modern web in a way that is possible without centralized servers. We don't need web3, we need internet2. > That is not to say that all p2p software is bad, especially since we call p2p a lot of things that are not entirely p2p. For example, BitTorrent is a p2p software, but its actual usage by humans relies on multiple more-or-less centralized point, trackers and torrent search engine. libp2p and scuttlebutt are pretty cool too. Both with their problems, but those problems seems solvable. Both seem more like internet2 than web3. p2p needs a new overlay network on top of the internet, just like the internet started as an overlay network on top of the telephony system.
Almondsetat 3y ago
Your software cannot be more decentralized than your hardware. For example, true p2p can only happen if you meet with someone and use a cable, bluetooth or local wifi. Anything over the internet needs to pass through routers and *poof* decentralization's gone and you now need to trust servers to a varying level of degrees
- colordrops 3y ago
  "varying" is a pretty wide range here. If you mean "trust" as in trust to maintain connectivity, yes, but beyond that there are established ways to create secure channels over untrusted networks. Could you provide specifics about what you mean if anything beyond basic connectivity?
  - Almondsetat 3y ago
    Sure: what is the process to start downloading a torrent? what is the process to message someone on Jami? what is the process to call someone on (old) Skype or Jitsi? Answer this and you realize you can only get as decentralized as your hardware infrastructure
    - Charon77 3y ago
      Well for torrent it starts by contacting one of the various trackers that know other peers. It's not centralized but there's only a couple of trackers out there. There's no trust between any of the peers, but each of the torrent piece has associated hash, meaning peers cannot give you invalid data without being caught (unless hash collision occurs). Peers can be discovered with DHT magic, but ultimately, can only be dialed if the ISP allows peers to receive connections.
- Barrin92 3y ago
  >Anything over the internet needs to pass through routers and poof decentralization's gone That's not true. Yes the strict p2p connection is gone but decentralization is what the name says, a network of connections without a single center. The internet and it's routing systems are decentralized. Of course every decentralized system can also be stressed to a point of failure and not every node is automatically trustworthy.
unethical_ban 3y ago
I'll add to what others have said better. Decentralized, globally accessible resources still take some kind of coordination to discover those resources, and a shit-ton of redundant nodes and data and routing. There is always some coordination or consensus. At least, that's my take on it. Does not tor have official records for exit and bridge nodes?

deleted 3y ago

[deleted]

pphysch 3y ago

True P2P networks don't scale, because every node has to store an (accurate if partial) representation of the whole network. Growing the network is easy, but growing every single node is virtually impossible (unless you control them all...). Any attempt to tackle that tends to increase centralization, e.g. in the form of routing authorities. And if you try to tackle centralization directly (like banning routers or something), you will often create an anti-centralization regulator, which is, you guessed it, another form of centralization. So your decentralized P2P network is either small and works good, medium and works not so good, or large and not actually decentralized. The best P2P networks know their limits and don't try to scale infinitely. For human-oriented network, Dunbar's Number (N=~150) is a decent rule of thumb; any P2P network larger than that almost certainly has some form of centralization (like trusted bootstrapping/coordination server addresses that are hard-coded in every client install, etc.)

KMag 3y ago
> True P2P networks don't scale, because every node has to store an (accurate if partial) representation of the whole network Former LimeWire dev here... which P2P networks use a fully meshed topology? LimeWire and other Gnutella clients just have a random mesh with a fixed number of (ultra)peers. If the network gets too large, then your constrained broadcast queries hit their hop count before reaching the edge of the network, but that seems fine. Last I checked, Freenet used a variation on a random mesh. Kademlia's routing tables take O(log(N)) space and traffic per-peer to maintain (so O(N log(N)) for global total network space and traffic). Same for Chord (though, twice as much traffic due to not using a symmetric distance metric like XOR). There are plenty of "True" (non-centralized) P2P networks that aren't fully meshed.
- sanity 3y ago
  Creator of Freenet here. Freenet[1] relies on peers self-organizing into a small world network[2]. Small world networks have the advantage of being able to find data in log N time where N is the network size, they're also completely decentralized, self-healing, and distribute load evenly across peers. The principle is similar to DHTs like Kademlia but more flexible and intuitive IMO, while having similar scaling characteristics. It's surprisingly common for people to confuse small world networks with "scale free networks", but scale free networks rely on a subset of highly connected peers which do a disproportionate amount of the work - which isn't truly decentralized. The new Freenet design incorporates adaptive learning into the routing algorithm. When a peer is deciding where to route a message, it predicts the response probability and time for each neighboring peer based on past performance and chooses the best. With conventional "greedy routing", peers choose the neighbor with a location closest to the data being retrieved. The new approach is adaptive to actual network performance. [1] Both the original Freenet from 1999 and the new sequel we're currently building - see https://freenet.org/ https://freenet.org/ for more. We hope to launch the network in the next few weeks. [2] https://en.wikipedia.org/wiki/Small-world_network https://en.wikipedia.org/wiki/Small-world_network
  - KMag 3y ago
    Thanks for the great work, Ian! As far a second-generation Freenet, I heard i2p started out as a proposed re-factoring and generalization of Freenet's encrypted transport layer. Are there any plans on using i2p to carry Freenet traffic?
    - sanity 3y ago
      Thank you :) I2P was created by someone who was previously involved with Freenet, but its design is a lot closer to Tor than to Freenet. Both I2P and Tor are anonymizing proxies, they allow services to be hidden, but they're still centralized. While they are quite different, there is enough overlap that running Freenet over I2P (or Tor) would be wildly inefficient and slow, so I wouldn't recommend it. Freenet is designed to run over UDP directly. The new Freenet is designed to allow the creation of completely decentralized services. Briefly, it's a global key-value store in which keys are webassembly code that specify what values are permitted under that key, and the conditions under which those values can be modified. This key-value store is observable, so anyone can subscribe to a key and be notified immediately if the value changes. This is just scratching the surface, for anyone interested in a much more comprehensive explanation of the new Freenet please see this talk I gave a few months ago: [1] You can also find a FAQ here: [2] [1] https://www.youtube.com/watch?v=yBtyNIqZios https://www.youtube.com/watch?v=yBtyNIqZios [2] https://freenet.org/faq https://freenet.org/faq

p4bl0 3y ago

The main thing is that "true" (in the sense of absolute) decentralization does not actually work. It doesn't work technically, and it doesn't work socially/politically. We always need some form of collective control over what's going on. We need moderation tools, we need to me able to make errors and fix them, etc. Centralized systems tend to be authoritarians, but the pursue of absolute decentralization always end up being very individualistic (and some kind of wrongly placed elitism). There are middle grounds: federated systems for example, like Mastodon or emails, actually work. That is not to say that all p2p software is bad, especially since we call p2p a lot of things that are not entirely p2p. For example, BitTorrent is a p2p software, but its actual usage by humans relies on multiple more-or-less centralized point, trackers and torrent search engine.

r3trohack3r 3y ago
I think you and I would have a very interesting conversation. I agree with a lot of your points, but not your conclusions. > We always need some form of collective control over what's going on. We need moderation tools I agree with this, and also think it is possible in peer-to-peer systems. Ideally the collective is self-governed. Particularly, when it comes to moderation, the closer the moderation controls are to being under the control of the user consuming the content, the more just the system. > , we need to me able to make errors and fix them, etc. Yes, 100%. Equity < Cryptography. It's far more important that your equity be rock solid than it is for your cryptography to be rock solid. If someone steals your property in a cryptographically sound way, equity should always trump cryptography. It's far more important that you have the title to your vehicle than it is that you have the keys to your vehicle. I feel like many p2p systems have gotten this one backwards. > There are middle grounds: federated systems for example, like Mastodon or emails, actually work. I consider these centralized systems, just N copies of the same problem. I don't feel like the power imbalance between the users and the system administrators are addressed in a sufficient way on the fediverse as it is currently implemented. These systems have a classist, hierarchical, system where system administrators belong to a privileged class while users are second class citizens on the web. --- I feel one of the issues in the current centralized architectures is equity. When you go about moving through the world, you generate a large volume of valuable property (your data). But, today, you give away nearly all equity in that data. Centralized providers accumulate equity in their user's data, and that equity is how they pay the bills. I do believe that, in a very meaningful way, equity and privacy are nearly synonymous when it comes to corporations respecting the privacy of their users. Just because Netflix delivers a video to your smartphone doesn't mean you can turn around and sell that video to your friend. You have been granted access to the video, but it is not your property. The inverse needs to be true too. Just because you share your viewing habits with Netflix doesn't mean they can sell that data to Warner Brothers, that's not their property (I mean, today it is, but it shouldn't be). If users had equity in their data, the data broker market as it exists today would be piracy. P2P systems have failed to create a world where humans, their content, and their devices are meaningfully addressable for the web in a way that expresses equity as a first class citizen. A decentralized world is possible, it's just not possible on the internet. The internet, as it exists, is insufficient for expressing the concepts of the modern web in a way that is possible without centralized servers. We don't need web3, we need internet2. > That is not to say that all p2p software is bad, especially since we call p2p a lot of things that are not entirely p2p. For example, BitTorrent is a p2p software, but its actual usage by humans relies on multiple more-or-less centralized point, trackers and torrent search engine. libp2p and scuttlebutt are pretty cool too. Both with their problems, but those problems seems solvable. Both seem more like internet2 than web3. p2p needs a new overlay network on top of the internet, just like the internet started as an overlay network on top of the telephony system.

Almondsetat 3y ago

Your software cannot be more decentralized than your hardware. For example, true p2p can only happen if you meet with someone and use a cable, bluetooth or local wifi. Anything over the internet needs to pass through routers and *poof* decentralization's gone and you now need to trust servers to a varying level of degrees

colordrops 3y ago
"varying" is a pretty wide range here. If you mean "trust" as in trust to maintain connectivity, yes, but beyond that there are established ways to create secure channels over untrusted networks. Could you provide specifics about what you mean if anything beyond basic connectivity?
- Almondsetat 3y ago
  Sure: what is the process to start downloading a torrent? what is the process to message someone on Jami? what is the process to call someone on (old) Skype or Jitsi? Answer this and you realize you can only get as decentralized as your hardware infrastructure
  - Charon77 3y ago
    Well for torrent it starts by contacting one of the various trackers that know other peers. It's not centralized but there's only a couple of trackers out there. There's no trust between any of the peers, but each of the torrent piece has associated hash, meaning peers cannot give you invalid data without being caught (unless hash collision occurs). Peers can be discovered with DHT magic, but ultimately, can only be dialed if the ISP allows peers to receive connections.
Barrin92 3y ago
>Anything over the internet needs to pass through routers and poof decentralization's gone That's not true. Yes the strict p2p connection is gone but decentralization is what the name says, a network of connections without a single center. The internet and it's routing systems are decentralized. Of course every decentralized system can also be stressed to a point of failure and not every node is automatically trustworthy.

unethical_ban 3y ago

I'll add to what others have said better. Decentralized, globally accessible resources still take some kind of coordination to discover those resources, and a shit-ton of redundant nodes and data and routing. There is always some coordination or consensus. At least, that's my take on it. Does not tor have official records for exit and bridge nodes?