3 ms·

> (1) Developer ease of use You can't fix it with developer tools. You'd need to completely rewrite the way many hundreds of thousands of diverse, entrenched,

> (1) Developer ease of use You can't fix it with developer tools. You'd need to completely rewrite the way many hundreds of thousands of diverse, entrenched, autonomous entities with a vested interest in your failure think about networking. Most users are behind NAT. Several thousand employees in an office or students in a dorm will have the same public IP address and will never, under any circumstances, administer the router where NATing takes place. For users who do not pay their own ISP bills, it's impossible. By design. You are not going to convince the entire IT management/security establishment that they should allow arbitrary connections to their users' machines. With unlimited resources and political power, you might be able to convince all the SOHO router manufacturers to include some kind of standardized port-forwarding API. It will be several years before all the old devices are replaced, though. But you might get to a point where there are peer-to-peer systems for nontechnical users who own their routers. You still need client-server for when people are on an institutional connection/mobile. There are no peer-to-peer connections between users behind firewalls or NAT. This is true even in BitTorrent - if you're behind a firewall, your only peers are people who are not behind firewalls (seedboxes in datacenters, geeks who went into their router config, and people who are wasting IPv4 space by handing out fully routable IPs to individual clients.)

api 12y ago
Most of this is incorrect. Look into hole punching as a starting point. BitTorrent, SIP VoIP phones, desktop Skype, etc. all do this. It's sort of a hack that became a semi de facto standard, but so is NAT. There are some percentage who cannot do p2p even with smart traversal techniques. In my system I have measured this to be <5% of total users, few enough that free relaying is basically free to provide. The hardest problems surround mobile battery life, quotas, and coarse grained and limited multitasking. These are also likely solvable. They may require a rethink at the encapsulated layer 3 level, like TCP proxy ACK or linear coding. Or maybe push notification. Still researching this.
- superuser2 12y ago
  >BitTorrent http://www.bittorrent.com/help/manual/chapter0203 http://www.bittorrent.com/help/manual/chapter0203 BitTorrent users who do not have port forwarding enabled do not communicate with each other. BitTorrent has a highly technical user base and chances are good that someone with the file you want actually has enabled port forwarding or is running a server in a datacenter (seedbox). That wouldn't work for mainstream messaging. >desktop Skype Firewalled Skype users do not talk directly to each other, they talk to Supernodes (servers) running on nodes that do have port forwarding / no firewall. Conscripting random PCs as servers drew public outrage and Microsoft eventually (citing scalability and reliability) made all the Supernodes server-class hardware sitting in Microsoft datacenters. >SIP VoIP phones If you are in an office and can receive calls on a SIP phone, it's because your organization runs (or pays someone to run) a central server called a gateway, usually but not always as part of a PBX. The most common open-source PBX is called Asterisk. People who call you from the outside are connecting to your phone, but to your organization's gateway. Your gateway then sits on a (V)LAN with your phone, or your phone maintains an outgoing connection to it. If my friend has her router configured but I do not, then (absent a gateway) I could call her but she couldn't call me. So you use central gateways Many VoIP phones come with VPN clients so that they can behave like peers to phones within the organization if deployed at employees' homes or in certain branch office configurations where bridging is not already in place. In this case, you still have centralization at the level of the VPN server.
  - api 12y ago
    It's very weird that the BitTorrent docs say that, since I have seen different in practice, both Transmission and uTorrent support uTP, a tcp-like udp protocol that can traverse NAT. There is something like a standard. Few products use this standard by name, but many use it's techniques. http://en.m.wikipedia.org/wiki/STUN http://en.m.wikipedia.org/wiki/STUN So you are mistaken. Either that or for years I have suffered from packet hallucinations and clinical delusions of NAT traversal, cause I am literally watching it happen now. :) Here's some scenarios I have recently personally tested using simple UDP hole punching: * Institutional/enterprise NAT: two universities, one large corporation, one hotel * Double NAT with virtual machine behind physical behind NAT * Double NAT on a cellular connection (tethering) * NAT behind "privacy tunnel" VPN * Several consumer cable modem routers But it certainly is hard. It involves either complex or clever solutions, and to field a reliable product there must be relay fallback for the percentage of users who cannot traverse. (Since it's a small percentage it's virtually free to provide that.) That fallback must be fast and transparent to the user, and done in a way that is maximally compatible. I do TCP over port 443, which is almost always open, and have considered literal HTTP encapsulation over 80 as a second fallback that would work even through proxies. It can be done. Like I said, I have a system in the field now seeing 90-95% success rate with direct connectivity. No firewall config is needed, nor are users conscripted to act as relays. Like I said elsewhere on the thread: coping with the limitations of mobile is harder than dealing with NAT. I think that can be done too but there are still question marks, especially around iOS. Idea is to do all the super hard stuff once and then make it available in a very nice SDK that's also backed by a service with enterprise SLA, support, etc. (All cheaper than handling all bandwidth yourself of course, not to mention lower latencies for most users.) Hide all the complexity from both the user and developer and offer something professional. There are P2P libraries out there but none that are professional, polished, or backed. Target would probably be games, video, and other more demanding applications. I was just using SnapChat as a simple example but it's probably not the best one.

api 12y ago

Most of this is incorrect. Look into hole punching as a starting point. BitTorrent, SIP VoIP phones, desktop Skype, etc. all do this. It's sort of a hack that became a semi de facto standard, but so is NAT. There are some percentage who cannot do p2p even with smart traversal techniques. In my system I have measured this to be <5% of total users, few enough that free relaying is basically free to provide. The hardest problems surround mobile battery life, quotas, and coarse grained and limited multitasking. These are also likely solvable. They may require a rethink at the encapsulated layer 3 level, like TCP proxy ACK or linear coding. Or maybe push notification. Still researching this.

superuser2 12y ago
>BitTorrent http://www.bittorrent.com/help/manual/chapter0203 http://www.bittorrent.com/help/manual/chapter0203 BitTorrent users who do not have port forwarding enabled do not communicate with each other. BitTorrent has a highly technical user base and chances are good that someone with the file you want actually has enabled port forwarding or is running a server in a datacenter (seedbox). That wouldn't work for mainstream messaging. >desktop Skype Firewalled Skype users do not talk directly to each other, they talk to Supernodes (servers) running on nodes that do have port forwarding / no firewall. Conscripting random PCs as servers drew public outrage and Microsoft eventually (citing scalability and reliability) made all the Supernodes server-class hardware sitting in Microsoft datacenters. >SIP VoIP phones If you are in an office and can receive calls on a SIP phone, it's because your organization runs (or pays someone to run) a central server called a gateway, usually but not always as part of a PBX. The most common open-source PBX is called Asterisk. People who call you from the outside are connecting to your phone, but to your organization's gateway. Your gateway then sits on a (V)LAN with your phone, or your phone maintains an outgoing connection to it. If my friend has her router configured but I do not, then (absent a gateway) I could call her but she couldn't call me. So you use central gateways Many VoIP phones come with VPN clients so that they can behave like peers to phones within the organization if deployed at employees' homes or in certain branch office configurations where bridging is not already in place. In this case, you still have centralization at the level of the VPN server.
- api 12y ago
  It's very weird that the BitTorrent docs say that, since I have seen different in practice, both Transmission and uTorrent support uTP, a tcp-like udp protocol that can traverse NAT. There is something like a standard. Few products use this standard by name, but many use it's techniques. http://en.m.wikipedia.org/wiki/STUN http://en.m.wikipedia.org/wiki/STUN So you are mistaken. Either that or for years I have suffered from packet hallucinations and clinical delusions of NAT traversal, cause I am literally watching it happen now. :) Here's some scenarios I have recently personally tested using simple UDP hole punching: * Institutional/enterprise NAT: two universities, one large corporation, one hotel * Double NAT with virtual machine behind physical behind NAT * Double NAT on a cellular connection (tethering) * NAT behind "privacy tunnel" VPN * Several consumer cable modem routers But it certainly is hard. It involves either complex or clever solutions, and to field a reliable product there must be relay fallback for the percentage of users who cannot traverse. (Since it's a small percentage it's virtually free to provide that.) That fallback must be fast and transparent to the user, and done in a way that is maximally compatible. I do TCP over port 443, which is almost always open, and have considered literal HTTP encapsulation over 80 as a second fallback that would work even through proxies. It can be done. Like I said, I have a system in the field now seeing 90-95% success rate with direct connectivity. No firewall config is needed, nor are users conscripted to act as relays. Like I said elsewhere on the thread: coping with the limitations of mobile is harder than dealing with NAT. I think that can be done too but there are still question marks, especially around iOS. Idea is to do all the super hard stuff once and then make it available in a very nice SDK that's also backed by a service with enterprise SLA, support, etc. (All cheaper than handling all bandwidth yourself of course, not to mention lower latencies for most users.) Hide all the complexity from both the user and developer and offer something professional. There are P2P libraries out there but none that are professional, polished, or backed. Target would probably be games, video, and other more demanding applications. I was just using SnapChat as a simple example but it's probably not the best one.