7 ms·

The Legacy of Peer-to-Peer Systems

98 points by CarlosBaquero 4y ago

CarlosBaquero 4y ago
What happened to peer-to-peer as a technological concept? Actually, we still use a lot of that technology.
- deleted 4y ago
  [deleted]
- superkuh 4y ago
  Smartphones took over as people's primary "computers" of choice. And mobile devices, generally, don't even get an IPv4 address with ports as most are behind carrier NAT. So most people cannot participate on the internet anymore and require third parties to hold their metaphorical hand when doing network operations. For people still using actual computers with real internet connections and ports p2p is still as big, and as useful, as ever. It's just that the relative percentage of online users with actual internet connections has shrunk. The absolute number of people with real computers and connections has not shrunk.
  - littlestymaar 4y ago
    Being behind a NAT poses constraints for p2p technologies (you need some well-known servers to do the hole-punching and act as a relay, but that's not too different from the well-know IPs that are needed for bootstraping a regular p2p system anyway, except of course, not every NAT are friendly to hole punching, and that's a problem as well…) but that also has a significant security and privacy advantage: since you aren't openly connected to the internet, you don't casually leak your computer's IP to the random strangers you're interacting with (at least when we're talking about a NAT you share with other people, not just your ISP box's NAT) and the amount of harm they can actually do to you is significantly lower. In the end I think the internet would actually be a significantly better place security-wise for p2p if IPs weren't directly routable by default, and NAT with all its limitations gives you mostly that.
    - dinosaurdynasty 4y ago
      NAT punching definitely tells other peers your NAT's IP address (and often your local address too, but that's less important). Unless you're behind CGNAT, your NAT IP can often be used to find your neighborhood with public information. With private information (a legal challenge for example) you can find the exact subscriber/house.
      - littlestymaar 4y ago
        > NAT punching definitely tells other peers your NAT's IP address Yes, and that's all you share, so when the NAT is shared with other people (like other students on a campus for instance, or other customers of your phone mobile phone carrier) the amount of info that can be collected is much lower than if you have a public IP address for your computer. > Unless you're behind CGNAT Did you read what I wrote above, when I said: “at least when we're talking about a NAT you share with other people, not just your ISP box's NAT”. > (and often your local address too, but that's less important). Here you're mixing up the hole-punching part with the signaling protocol (ICE, which have had this issue in the past, before browsers switched to mDNS[1] instead of private IP addresses in ICE candidates). [1]: https://groups.google.com/g/discuss-webrtc/c/6stQXi72BEU?pli=1 https://groups.google.com/g/discuss-webrtc/c/6stQXi72BEU?pli...
        dinosaurdynasty 4y ago
        You need a signaling protocol to do hole punching.
        littlestymaar 4y ago
        The two are working together to establish a p2p connection behind a NAT but that doesn't make them equivalent. It's like saying “UDP sometimes leaks your local IP address”, that's factually inaccurate.
  - ztetranz 4y ago
    Here's an off-topic but somewhat related question that I've been meaning to ask somewhere. How do "plug and play" consumer devices that receive an incoming call / connection work behind the typical home NAT router? I have an OOMA VOIP phone service which is plugged into my home router with no ports forwarded. It has no trouble receiving an incoming call. Does it simply open an outgoing connection and hold it open indefinitely?
    - aliqot 4y ago
      STUN or an intermediary
    - wmf 4y ago
      Yes, that's pretty much the only way it could work.
- dottedmag 4y ago
  It became... boring technology.
  - throwaway14356 4y ago
    haha, so many tb's now for such small money
- sliken 4y ago
  Years ago, most computing devices were desktops. They often had a routable IP address, unlimited power, and would happily sit passing packets all day. This made things like a DHT practical, so you could find your other peers. This made things like the early days of skype where except for auth, chat and file sharing was p2p. After being online for long enough and having a routable IP, you could become a supernode to help less fortunate nodes talk to each other. These days a much larger fraction of computing devices are on battery, on expensive networks like cellular, and can't really tolerate being part of a DHT. Increasing use of NAT/Masquerading makes a harder (and a support nightmare) to accept incoming packets from new peers. One solution to this is to add a "superpeer" to a router distribution like OpenWRT, or sell the "plug/wallwart" to help. That way a cheap (under $100) computer could build reputation with it's peers, accept incoming packets form new peers, provide some storage, and keep up with DHT maintenance. Then low power and/or expensive network peers could just check their "home" superpeer and get what they need quickly with minimal bandwidth and power.
  - pbronez 4y ago
    I agree that a pool of equal peers is tough these days. I think the Fediverse has a pretty good approach, where most end users are on mobile but you can spin up a server whenever. There’s still a big complexity/skill/cost jump from “I toot from my iPhone” to “I run a mastodon instance for my company” though. Some of that can be addressed by managed hosting. It’s probably preferable to have a “super peer” though. In my mind, a superpeer runs the same software as a peer, but does more work because it can. It should be easier to maintain than a full server. I’m taking about the difference between: A) manage a mastodon node, with its own redis, PostgreSQL, web server, object storage, etc And B) run BitTorrent in the background on your gaming PC to seed the latest cut of the niche documentary you’re working on There’s a lot of interesting self-hosting projects happening, but they tend to focus on helping you run kubernetes or a similar container orchestrator. That’s still way more complex than an executable. I think we need things to get a bit more opinionated again…
    - sliken 4y ago
      Agreed. Supernodes should be near zero admin, and be able to run where advantageous, not just where an expert is available. OwnCloud/Nextcloud running every more services and at least looking at fediverse integration. Various NASs allow local applications for photos, plex, etc. All targeted at being nearly turnkey and friendly to the average consumer. Ideally things become easy enough that people can depend less on FAANG and do more with services that are distributed. Hopefully the software can get to the point where your server helps when it can, but if it's down other servers would help out. I just bought a $120 ($140 with case) router, 8 cores, 8GB ram, 32GB storage, GigE + 2x2.5gbe. I've love to dedicate it for messaging, filesharing, mastodon, photo storage/viewing/sharing, etc and even have it exchange services with others so that message/photos/whatever can be shared even if it goes down. Hopefully it gets there, pretty amazing resources are available cheap these days. I'd happily trade 2/3rd of my resources for other peers ... if they did the same for me.
- matheusmoreira 4y ago
  NAT happened. Everyone is behind a NAT these days. No way to directly network with any computer on the internet anymore.
- lisbon44 4y ago
  For 'Linux ISOs' some of the decline was bandwidth getting cheaper for mirror hosts, meaning that in place of P2P seeders there are now well-connected mirrors that are not 'bandwidth handicapped' in comparison to the past
crosser 4y ago
> Coin generation and distribution, particularly when coins can be traded for fiat currency or goods, creates an incentive mechanism to keep the P2P system running. Ability to trade for fiat currency proved to be a mixed blessing at best, and a downfall at worst, of this approach. When a system is so lucrative as a vehicle for ponzi schemes, it inevitably gets hijacked, and becomes unable to serve its declared purpose.
AdmiralAsshat 4y ago
P2P still works great. Unfortunately the well has been poisoned for alot of people, as it's widely assumed that "torrenting" of any sort must be illegal, even if it's for perfectly legitimate purposes.
- aliqot 4y ago
  Nothing illegal about Linux ISOs
  - oriolid 4y ago
    Linux ISOs or "Linux ISOs"?
- tartoran 4y ago
  Just wanted to say this. Still using p2p in 2022. Yes, it’s less frequent than before due to ubiquitous content online but it’s still very reliable.
  - DoingIsLearning 4y ago
    Where does one go and search for magnet URIs in 2022? (Not sure if this is a copyright sensitve question but curious)
    - kh_hk 4y ago
      This is neat and clean https://torrents-csv.ml https://torrents-csv.ml
    - noman-land 4y ago
      BTDigg is another option. https://en.wikipedia.org/wiki/BTDigg https://en.wikipedia.org/wiki/BTDigg
- RunningDroid 4y ago
  > it's widely assumed that "torrenting" of any sort must be illegal, even if it's for perfectly legitimate purposes. My understanding is that many people consider piracy illegal in the "corporations paid to make it illegal because it threatened their business model" sense
deleted 4y ago
[deleted]
musicale 4y ago
I think part of what has happened to decentralized internet systems is technological, and part of it is terminology. It seems like we still have decentralized systems but now call them fediverse, distributed, decentralized, federated, ipfs, web 3.0, etc.. It might have been interesting if he'd looked at searches for those terms as well.
- ilyt 4y ago
  They aren't really decentralized in old meaning. It's more a bunch of fiefdoms than something actually decentralized. Because at some point you need to deal with trolls/spammers at the gates
api 4y ago
ZeroTier is peer to peer but we don’t end up using the term very much. Loads of conferencing systems use P2P. WebRTC is P2P. Yet the term isn’t used there too much either. Same with games. I think the term has just faded from the hype cycle.
austin-cheney 4y ago
I am writing a Node based P2P and messaging app for privacy. It’s mostly for linking personal devices as a single shared file system but it also has a security model for restricted sharing between trusted friends. https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems The application concept is difficult to explain so I have started calling it an operating system as the application is getting larger.
amingilani 4y ago
ELI5, why can’t we have a combination of modern P2P like WebRTC and onion routing to anonymize source and destinations?
- wmf 4y ago
  That should be possible. It would be relatively inefficient to chain through multiple browsers.
- Sean-Der 4y ago
  That does exist today https://snowflake.torproject.org/ https://snowflake.torproject.org/! You can use WebRTC in the browser, or the Go implementation https://github.com/keroserene/snowflake https://github.com/keroserene/snowflake
throwawaaarrgh 4y ago
ISPs killed P2P. Even if you have the upstream to seed all day, they'll just rate-limit you. There is no monetary incentive for them to allow P2P anyway. That said.... Gnutella/Gnutella2, eDonkey2000, and others were objectively crappy designs, but still worked very well at one use case: distributing rare files. The thing is, if you have a stable high-speed mirror, the files aren't rare anymore. Hence the only files that are rare are the illegal ones, or ones that can't find a mirror to host them. There's just not much point to P2P. Sometimes there's good reason to distribute illegal files, like getting past state censorship. But the whole world isn't going to adopt a wonky solution for a rare use-case. Hence P2P has not & will not take off.
- Reventlov 4y ago
  > ISPs killed P2P. Even if you have the upstream to seed all day, they'll just rate-limit you. There is no monetary incentive for them to allow P2P anyway. I don't know where you live, but in the EU we tend to take the net neutrality seriously, and that does not happen.
  - Karrot_Kream 4y ago
    In the US, ISPs offer lower uplink speeds than downlink speeds as a cost cutting measure. More modem channels will be dedicated to downlink than uplink. With fiber ISPs this is beginning to change, but ISP quality in the US quality is highly variable.
    - deleted 4y ago
      [deleted]
    - dale_glass 4y ago
      It's not so much a cost cutting measure as a competition measure. A given channel has only so much bandwidth. In DSL and the like, you can pick which part is dedicated to up, and which to down because you only have one pair of wires to work with. So your link gives you say, 100 Mbps. You can split that 50/50, but then your competition can go with a 90/10 split, and look, their downloads are much faster!

98 points by CarlosBaquero 4y ago

CarlosBaquero 4y ago

What happened to peer-to-peer as a technological concept? Actually, we still use a lot of that technology.

deleted 4y ago
[deleted]
superkuh 4y ago
Smartphones took over as people's primary "computers" of choice. And mobile devices, generally, don't even get an IPv4 address with ports as most are behind carrier NAT. So most people cannot participate on the internet anymore and require third parties to hold their metaphorical hand when doing network operations. For people still using actual computers with real internet connections and ports p2p is still as big, and as useful, as ever. It's just that the relative percentage of online users with actual internet connections has shrunk. The absolute number of people with real computers and connections has not shrunk.
- littlestymaar 4y ago
  Being behind a NAT poses constraints for p2p technologies (you need some well-known servers to do the hole-punching and act as a relay, but that's not too different from the well-know IPs that are needed for bootstraping a regular p2p system anyway, except of course, not every NAT are friendly to hole punching, and that's a problem as well…) but that also has a significant security and privacy advantage: since you aren't openly connected to the internet, you don't casually leak your computer's IP to the random strangers you're interacting with (at least when we're talking about a NAT you share with other people, not just your ISP box's NAT) and the amount of harm they can actually do to you is significantly lower. In the end I think the internet would actually be a significantly better place security-wise for p2p if IPs weren't directly routable by default, and NAT with all its limitations gives you mostly that.
  - dinosaurdynasty 4y ago
    NAT punching definitely tells other peers your NAT's IP address (and often your local address too, but that's less important). Unless you're behind CGNAT, your NAT IP can often be used to find your neighborhood with public information. With private information (a legal challenge for example) you can find the exact subscriber/house.
    - littlestymaar 4y ago
      > NAT punching definitely tells other peers your NAT's IP address Yes, and that's all you share, so when the NAT is shared with other people (like other students on a campus for instance, or other customers of your phone mobile phone carrier) the amount of info that can be collected is much lower than if you have a public IP address for your computer. > Unless you're behind CGNAT Did you read what I wrote above, when I said: “at least when we're talking about a NAT you share with other people, not just your ISP box's NAT”. > (and often your local address too, but that's less important). Here you're mixing up the hole-punching part with the signaling protocol (ICE, which have had this issue in the past, before browsers switched to mDNS[1] instead of private IP addresses in ICE candidates). [1]: https://groups.google.com/g/discuss-webrtc/c/6stQXi72BEU?pli=1 https://groups.google.com/g/discuss-webrtc/c/6stQXi72BEU?pli...
      - dinosaurdynasty 4y ago
        You need a signaling protocol to do hole punching.
        littlestymaar 4y ago
        The two are working together to establish a p2p connection behind a NAT but that doesn't make them equivalent. It's like saying “UDP sometimes leaks your local IP address”, that's factually inaccurate.
- ztetranz 4y ago
  Here's an off-topic but somewhat related question that I've been meaning to ask somewhere. How do "plug and play" consumer devices that receive an incoming call / connection work behind the typical home NAT router? I have an OOMA VOIP phone service which is plugged into my home router with no ports forwarded. It has no trouble receiving an incoming call. Does it simply open an outgoing connection and hold it open indefinitely?
  - aliqot 4y ago
    STUN or an intermediary
  - wmf 4y ago
    Yes, that's pretty much the only way it could work.
dottedmag 4y ago
It became... boring technology.
- throwaway14356 4y ago
  haha, so many tb's now for such small money
sliken 4y ago
Years ago, most computing devices were desktops. They often had a routable IP address, unlimited power, and would happily sit passing packets all day. This made things like a DHT practical, so you could find your other peers. This made things like the early days of skype where except for auth, chat and file sharing was p2p. After being online for long enough and having a routable IP, you could become a supernode to help less fortunate nodes talk to each other. These days a much larger fraction of computing devices are on battery, on expensive networks like cellular, and can't really tolerate being part of a DHT. Increasing use of NAT/Masquerading makes a harder (and a support nightmare) to accept incoming packets from new peers. One solution to this is to add a "superpeer" to a router distribution like OpenWRT, or sell the "plug/wallwart" to help. That way a cheap (under $100) computer could build reputation with it's peers, accept incoming packets form new peers, provide some storage, and keep up with DHT maintenance. Then low power and/or expensive network peers could just check their "home" superpeer and get what they need quickly with minimal bandwidth and power.
- pbronez 4y ago
  I agree that a pool of equal peers is tough these days. I think the Fediverse has a pretty good approach, where most end users are on mobile but you can spin up a server whenever. There’s still a big complexity/skill/cost jump from “I toot from my iPhone” to “I run a mastodon instance for my company” though. Some of that can be addressed by managed hosting. It’s probably preferable to have a “super peer” though. In my mind, a superpeer runs the same software as a peer, but does more work because it can. It should be easier to maintain than a full server. I’m taking about the difference between: A) manage a mastodon node, with its own redis, PostgreSQL, web server, object storage, etc And B) run BitTorrent in the background on your gaming PC to seed the latest cut of the niche documentary you’re working on There’s a lot of interesting self-hosting projects happening, but they tend to focus on helping you run kubernetes or a similar container orchestrator. That’s still way more complex than an executable. I think we need things to get a bit more opinionated again…
  - sliken 4y ago
    Agreed. Supernodes should be near zero admin, and be able to run where advantageous, not just where an expert is available. OwnCloud/Nextcloud running every more services and at least looking at fediverse integration. Various NASs allow local applications for photos, plex, etc. All targeted at being nearly turnkey and friendly to the average consumer. Ideally things become easy enough that people can depend less on FAANG and do more with services that are distributed. Hopefully the software can get to the point where your server helps when it can, but if it's down other servers would help out. I just bought a $120 ($140 with case) router, 8 cores, 8GB ram, 32GB storage, GigE + 2x2.5gbe. I've love to dedicate it for messaging, filesharing, mastodon, photo storage/viewing/sharing, etc and even have it exchange services with others so that message/photos/whatever can be shared even if it goes down. Hopefully it gets there, pretty amazing resources are available cheap these days. I'd happily trade 2/3rd of my resources for other peers ... if they did the same for me.
matheusmoreira 4y ago
NAT happened. Everyone is behind a NAT these days. No way to directly network with any computer on the internet anymore.
lisbon44 4y ago
For 'Linux ISOs' some of the decline was bandwidth getting cheaper for mirror hosts, meaning that in place of P2P seeders there are now well-connected mirrors that are not 'bandwidth handicapped' in comparison to the past

crosser 4y ago

> Coin generation and distribution, particularly when coins can be traded for fiat currency or goods, creates an incentive mechanism to keep the P2P system running. Ability to trade for fiat currency proved to be a mixed blessing at best, and a downfall at worst, of this approach. When a system is so lucrative as a vehicle for ponzi schemes, it inevitably gets hijacked, and becomes unable to serve its declared purpose.

AdmiralAsshat 4y ago

P2P still works great. Unfortunately the well has been poisoned for alot of people, as it's widely assumed that "torrenting" of any sort must be illegal, even if it's for perfectly legitimate purposes.

aliqot 4y ago
Nothing illegal about Linux ISOs
- oriolid 4y ago
  Linux ISOs or "Linux ISOs"?
tartoran 4y ago
Just wanted to say this. Still using p2p in 2022. Yes, it’s less frequent than before due to ubiquitous content online but it’s still very reliable.
- DoingIsLearning 4y ago
  Where does one go and search for magnet URIs in 2022? (Not sure if this is a copyright sensitve question but curious)
  - kh_hk 4y ago
    This is neat and clean https://torrents-csv.ml https://torrents-csv.ml
  - noman-land 4y ago
    BTDigg is another option. https://en.wikipedia.org/wiki/BTDigg https://en.wikipedia.org/wiki/BTDigg
RunningDroid 4y ago
> it's widely assumed that "torrenting" of any sort must be illegal, even if it's for perfectly legitimate purposes. My understanding is that many people consider piracy illegal in the "corporations paid to make it illegal because it threatened their business model" sense

deleted 4y ago

[deleted]

musicale 4y ago

I think part of what has happened to decentralized internet systems is technological, and part of it is terminology. It seems like we still have decentralized systems but now call them fediverse, distributed, decentralized, federated, ipfs, web 3.0, etc.. It might have been interesting if he'd looked at searches for those terms as well.

ilyt 4y ago
They aren't really decentralized in old meaning. It's more a bunch of fiefdoms than something actually decentralized. Because at some point you need to deal with trolls/spammers at the gates

api 4y ago

ZeroTier is peer to peer but we don’t end up using the term very much. Loads of conferencing systems use P2P. WebRTC is P2P. Yet the term isn’t used there too much either. Same with games. I think the term has just faded from the hype cycle.

austin-cheney 4y ago

I am writing a Node based P2P and messaging app for privacy. It’s mostly for linking personal devices as a single shared file system but it also has a security model for restricted sharing between trusted friends. https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems The application concept is difficult to explain so I have started calling it an operating system as the application is getting larger.

amingilani 4y ago

ELI5, why can’t we have a combination of modern P2P like WebRTC and onion routing to anonymize source and destinations?

wmf 4y ago
That should be possible. It would be relatively inefficient to chain through multiple browsers.
Sean-Der 4y ago
That does exist today https://snowflake.torproject.org/ https://snowflake.torproject.org/! You can use WebRTC in the browser, or the Go implementation https://github.com/keroserene/snowflake https://github.com/keroserene/snowflake

throwawaaarrgh 4y ago

ISPs killed P2P. Even if you have the upstream to seed all day, they'll just rate-limit you. There is no monetary incentive for them to allow P2P anyway. That said.... Gnutella/Gnutella2, eDonkey2000, and others were objectively crappy designs, but still worked very well at one use case: distributing rare files. The thing is, if you have a stable high-speed mirror, the files aren't rare anymore. Hence the only files that are rare are the illegal ones, or ones that can't find a mirror to host them. There's just not much point to P2P. Sometimes there's good reason to distribute illegal files, like getting past state censorship. But the whole world isn't going to adopt a wonky solution for a rare use-case. Hence P2P has not & will not take off.

Reventlov 4y ago
> ISPs killed P2P. Even if you have the upstream to seed all day, they'll just rate-limit you. There is no monetary incentive for them to allow P2P anyway. I don't know where you live, but in the EU we tend to take the net neutrality seriously, and that does not happen.
- Karrot_Kream 4y ago
  In the US, ISPs offer lower uplink speeds than downlink speeds as a cost cutting measure. More modem channels will be dedicated to downlink than uplink. With fiber ISPs this is beginning to change, but ISP quality in the US quality is highly variable.
  - deleted 4y ago
    [deleted]
  - dale_glass 4y ago
    It's not so much a cost cutting measure as a competition measure. A given channel has only so much bandwidth. In DSL and the like, you can pick which part is dedicated to up, and which to down because you only have one pair of wires to work with. So your link gives you say, 100 Mbps. You can split that 50/50, but then your competition can go with a 90/10 split, and look, their downloads are much faster!