7 ms·

TCP simultaneous open and peer to peer networking

95 points by jonchang 13y ago

dfc 13y ago
TCP/IP Illustrated Vol. 1, Stevens, pg: 250-252
PurplePanda 13y ago
This is exactly what one of those UDP "weird hole punching tricks" does. Except of course without connections, it not being TCP. nat-traverse (http://m19s28.dyndns.org/iblech/nat-traverse/ http://m19s28.dyndns.org/iblech/nat-traverse/) works this way as I understand it.
- syassami 13y ago
  here's an excellent paper on hole punching/nat traversal http://www.brynosaurus.com/pub/net/p2pnat/ http://www.brynosaurus.com/pub/net/p2pnat/
  - petercooper 13y ago
    For anyone who wants to fiddle as well, I wrote some very simple Ruby code several years ago to do it too: http://www.rubyinside.com/skype-style-firewall-busting-with-ruby-and-udp-399.html http://www.rubyinside.com/skype-style-firewall-busting-with-...
phlo 13y ago
STUN[1] does a very similar thing, including some trickery to get through all kinds of plastic boxes messing with your packets. AeroFS[2] is another, interesting approach. It's basically a dropbox clone where all data remains on the user's computer. Transfers are encrypted with TLS and tunneled through the company's servers. Another interesting thing might be Opera Unite[3]. It's an in-browser web server that can be souped up using various extensions like photo sharing, games or chat. As far as I know its NAT traversal is somewhat limited, though. [1] http://en.wikipedia.org/wiki/Session_Traversal_Utilities_for_NAT http://en.wikipedia.org/wiki/Session_Traversal_Utilities_for... [2] https://aerofs.com/ https://aerofs.com/ [3] http://unite.opera.com/applications/ http://unite.opera.com/applications/
- nwf 13y ago
  While on the topic of clever UDP-based NAT traversal techniques, I think my favorite to date is pwnat [http://samy.pl/pwnat/ http://samy.pl/pwnat/]. Uses ICMP in a clever way, requires no 3rd party.
- throwawaykf02 13y ago
  Note that STUN itself does not do traversal for you - it simply gives you enough information to do it yourself.
mixedbit 13y ago
As the author notes, for such hack to have any chances of working reliably a third party is needed to synchronize connection setup. But a third party with a public IP could also forward an encrypted TCP connection between two machines that do not have public IP addresses. Such approach does not require any hacks and gives a strong guarantee that the third party can not snoop the connection. The only thing that the third party learns is that there is a connection, but the proposes solution also has this drawback.
- idupree 13y ago
  That's standardized as TURN, if I'm not mistaken ( https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_... ). Other NAT traversal mechanisms (STUN) don't make the server pay for bandwidth. (Which methods work with TCP, in practice?)
  - dfc 13y ago
    There was a good thread on p2p-hackers a little while back. I apologize for not digging up the link but I'm on my way to bed. I think the thread was something like "state of the art NAT traversal"
- ars 13y ago
  Yes, but your approach requires a lot of bandwidth on the 3rd party. But a 3rd party doing nothing except helping with synchronization uses barely any bandwidth and someone could easily host it for free on some server that is not otherwise fully used, simply as a service to the public.
- amenod 13y ago
  I have to disagree on that - can you really trust encrpytion? Are you sure? On the other hand, why depend on (supposedly) strong encryption against 3rd party if you can just avoid passing the message through their hands altogether? This seems like a great stuff if it works. Unbeleviable I was able to avoid knowing about it until now. Kudos to top poster, great info.
gcb0 13y ago
Why not just adopt ipv6 and connect to the box directly and intuitively when all ipv4 nat boxes in the middle die?
- hvidgaard 13y ago
  NAT'ing is such an ingrained way of doing things, and it also happens to be a great way to separate your network from the rest of the network/internet. I don't think it'll go away any time soon.
  - tjgq 13y ago
    Please allow me to be pedantic here. What actually separates your network from the rest of the Internet is called a firewall. It is unfortunate that common usage has conflated the two terms to the point that many people believe they must be inside a NAT to be firewalled from the outside. It's actually one of many misunderstandings related to the IPv6 transition.
    - hvidgaard 13y ago
      Don't get me wrong, I know the difference between NAT and a firewall, but NAT'ing automatically (and by definition) hides the network behind, firewall or not.
      - phaker 13y ago
        Then you are using NAT as a simple firewall. You could have the firewall part without the translation part (or with deterministic two-way translation), which would eliminate 99% of problems. let me explain: A NAT "hides the network" behind it because it does two things: 1. drops incoming connections (or more precisely, incoming traffic when there was no outgoing traffic from that ip:port on that protocol) 2. obscures internal network addresses Dropping incoming connections is what makes it a simple firewall and you could set up a "real" firewall to do the same (there are probably products doing that out of the box). Most firewalls are more complex than that, but they don't _have to be_. What makes NATs different is the address translation that obscures the structure of the network behind it. Traditionally this is needed because you have lots of internal addresses and few external addresses (often just one). But if you have more hosts than external IPs then you can't have a static mapping without collisions because you have more internal ip:port pairs than external ip:port pairs. Which is why NAT generates internal-external mappings as it goes and why old mappings need to be discarded and why it needs to drop incoming connections (it can't know where to forwarded it to if there is no mapping). THIS is the part people hate, because it means you can't reliably connect two NATed hosts. If hole punching worked reliably it wouldn't be a problem, it feels like a hack, but the internet is a patchwork of much worse hacks.
peterwwillis 13y ago
http://nmap.org/misc/split-handshake.pdf http://nmap.org/misc/split-handshake.pdf
ricardobeat 13y ago
Isn't this called "NAT punching", and generally used by games and other sw to establish peer-to-peer connections?
- DennisP 13y ago
  That's usually UDP. With this method, you "get all of the utility [TCP] provides without trying to reproduce it over UDP."
chacham15 13y ago
From my understanding a NAT is going to remap an internal port to a different external port and upon receiving a RST will delete the mapping. Is this not correct? Also, doesnt connection reversal solve this problem? (Connection reversal: have A connect to intermediary S. have B connect to intermediary S. S sends A B's info and vice versa. A connects to B before closing connection with S.) This also does not require S to forward data, only connection information. Am I missing something there?
- digitalsushi 13y ago
  There are like 4 different NAT port translation models. In some of the models, the NAT algorithm will actually preserve the ephemeral port number in the NAT mapping. This falls apart quickly when you have a few NAT clients. But it can make legacy software work a lot better. We have weeded most of that software out at this point, or at least provided application gateways that inspect the content of the packet and tweak little things - consider how you need an extra kernel module for FTP and SIP and stuff. The models are detailed nicely in this diagram. https://en.wikipedia.org/wiki/Network_address_translation#Methods_of_port_translation https://en.wikipedia.org/wiki/Network_address_translation#Me...
latitude 13y ago
This works and it works reasonably well. I had used simultaneous-open TCP punching as a fallback for UDP punching when I was doing [0] and it did help in 10-20% of cases when (a fairly elaborate version of) UDP punching failed. That's on a scale of several hundred thousand mediated connections per day. One caveat though is that it requires implementing a bot-like functionality in the clients, meaning that a mediating server should be able to tell a client - "create a socket, bind it to this ip:port, wait, wait... connect to that ip:port". Obviously, this is an ideal platform for DDoS attacks if someone ever manages to re-point clients to a rogue mediation server. So, yeah, it works well, but there are some not so obvious trade-offs. [0] http://swapped.cc/hamachi http://swapped.cc/hamachi
- steeve 13y ago
  Ah! So that's how you guys did it! I always wondered what you were doing beyond UDP hole punching.
- malloc2x 13y ago
  Is there a Free software package that does something like this today?
  - synctext 13y ago
    TCP simultaneous open is very difficult to use. In our experience with a deployed P2P system, UDP hole punching is more successful with all the strange NAT boxes deployed. Our success rate is 95.3% now in the wild. As said above, this TCP fallback sounds like an excellent way to get to 97% connection success rate! Note that there is an upcoming IETF Internet Standard describing UDP hole punching: http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-06#section-3.10 http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-06#... We've implemented this as LGPL code: svn.tribler.org/libswift/branches/ppsp-03/
eridius 13y ago
Because nobody has linked it yet, the referenced XKCD is http://xkcd.com/949/ http://xkcd.com/949/
maffydub 13y ago
I'm surprised this works... or at least works reliably. I understood many simple firewalls just blocked inbound SYNs (without performing any kind of IP address lookup). This would obviously prevent this mechanism from working. Is this not (or no longer) the case?
- amenod 13y ago
  I am not an expert in this area, but from what I understand, firewalls keep a list of outbound connections. If inbound connection comes from a known destination IP+port, it will be forwarded to the internal "source" IP+port. This is how I understand it: if both A and B are behind firewalls, they use C to reach an agreement about IPs and ports used. Then A sends a packet to B, which is silently dropped at B's firewall. Then B send a packet to A - since it looks like an "answer" to previous request it is forwarded by A's firewall to A. Then A sends another packet to B, which is also forwarded by B's firewall to B. Voila, connection made. :) Note that this is just my understanding, so I would appreciate if someone more knowledgeable in this area would chime in.
wtracy 13y ago
Is this the sort of technology that P2P systems like BitTorrent use? I've never understood how BT works in setups that block incoming connections.
AndreasFrom 13y ago
BitTorrent Sync [1] can help with the explicit file sharing problem from the xkcd. Too bad it's not open source. [1]: http://labs.bittorrent.com/experiments/sync.html http://labs.bittorrent.com/experiments/sync.html
czzarr 13y ago
what was the title of the book you read?
throwawaykf02 13y ago
I'm not sure what OP has against UPnP port mapping. It's not an ideal option and certainly not the best-designed specification, but it's a way to explicitly tell the NAT to do what we want, rather than having to trick it into doing what we want. Also, one option not mentioned here yet is Teredo: http://en.m.wikipedia.org/wiki/Teredo_tunneling http://en.m.wikipedia.org/wiki/Teredo_tunneling It's primarily an IPv6 transition technology, but it comes with NAT traversal, hole-punching and all that built-in. What you get is a virtual IPv6 network interface to which you simply bind your socket, and you can connect to other Teredo / IPv6 sockets... if the whimsies of the Gods of Middleboxes allow. What makes it attractive is that it is deployed on all Windows machines WinXP and later (and enabled by default on Vista and later), giving it a huge deployment base. It's not present on non-Windows machines, of course, but there is a liberally licensed implementation for OSX and Linux called Miredo. uTorrent is one popular application that uses Teredo. However, Teredo tunneling does not work very reliably (apparently, its designers traded off connectivity for additional security), and it would be unadvisable to have that as your only NAT traversal method. But I think getting that option with minimal additional code is not a bad deal.

95 points by jonchang 13y ago

dfc 13y ago

TCP/IP Illustrated Vol. 1, Stevens, pg: 250-252

This is exactly what one of those UDP "weird hole punching tricks" does. Except of course without connections, it not being TCP. nat-traverse (http://m19s28.dyndns.org/iblech/nat-traverse/ http://m19s28.dyndns.org/iblech/nat-traverse/) works this way as I understand it.

syassami 13y ago
here's an excellent paper on hole punching/nat traversal http://www.brynosaurus.com/pub/net/p2pnat/ http://www.brynosaurus.com/pub/net/p2pnat/
- petercooper 13y ago
  For anyone who wants to fiddle as well, I wrote some very simple Ruby code several years ago to do it too: http://www.rubyinside.com/skype-style-firewall-busting-with-ruby-and-udp-399.html http://www.rubyinside.com/skype-style-firewall-busting-with-...

phlo 13y ago

STUN[1] does a very similar thing, including some trickery to get through all kinds of plastic boxes messing with your packets. AeroFS[2] is another, interesting approach. It's basically a dropbox clone where all data remains on the user's computer. Transfers are encrypted with TLS and tunneled through the company's servers. Another interesting thing might be Opera Unite[3]. It's an in-browser web server that can be souped up using various extensions like photo sharing, games or chat. As far as I know its NAT traversal is somewhat limited, though. [1] http://en.wikipedia.org/wiki/Session_Traversal_Utilities_for_NAT http://en.wikipedia.org/wiki/Session_Traversal_Utilities_for... [2] https://aerofs.com/ https://aerofs.com/ [3] http://unite.opera.com/applications/ http://unite.opera.com/applications/

nwf 13y ago
While on the topic of clever UDP-based NAT traversal techniques, I think my favorite to date is pwnat [http://samy.pl/pwnat/ http://samy.pl/pwnat/]. Uses ICMP in a clever way, requires no 3rd party.
throwawaykf02 13y ago
Note that STUN itself does not do traversal for you - it simply gives you enough information to do it yourself.

mixedbit 13y ago

As the author notes, for such hack to have any chances of working reliably a third party is needed to synchronize connection setup. But a third party with a public IP could also forward an encrypted TCP connection between two machines that do not have public IP addresses. Such approach does not require any hacks and gives a strong guarantee that the third party can not snoop the connection. The only thing that the third party learns is that there is a connection, but the proposes solution also has this drawback.

idupree 13y ago
That's standardized as TURN, if I'm not mistaken ( https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_... ). Other NAT traversal mechanisms (STUN) don't make the server pay for bandwidth. (Which methods work with TCP, in practice?)
- dfc 13y ago
  There was a good thread on p2p-hackers a little while back. I apologize for not digging up the link but I'm on my way to bed. I think the thread was something like "state of the art NAT traversal"
ars 13y ago
Yes, but your approach requires a lot of bandwidth on the 3rd party. But a 3rd party doing nothing except helping with synchronization uses barely any bandwidth and someone could easily host it for free on some server that is not otherwise fully used, simply as a service to the public.
amenod 13y ago
I have to disagree on that - can you really trust encrpytion? Are you sure? On the other hand, why depend on (supposedly) strong encryption against 3rd party if you can just avoid passing the message through their hands altogether? This seems like a great stuff if it works. Unbeleviable I was able to avoid knowing about it until now. Kudos to top poster, great info.

gcb0 13y ago

Why not just adopt ipv6 and connect to the box directly and intuitively when all ipv4 nat boxes in the middle die?

hvidgaard 13y ago
NAT'ing is such an ingrained way of doing things, and it also happens to be a great way to separate your network from the rest of the network/internet. I don't think it'll go away any time soon.
- tjgq 13y ago
  Please allow me to be pedantic here. What actually separates your network from the rest of the Internet is called a firewall. It is unfortunate that common usage has conflated the two terms to the point that many people believe they must be inside a NAT to be firewalled from the outside. It's actually one of many misunderstandings related to the IPv6 transition.
  - hvidgaard 13y ago
    Don't get me wrong, I know the difference between NAT and a firewall, but NAT'ing automatically (and by definition) hides the network behind, firewall or not.
    - phaker 13y ago
      Then you are using NAT as a simple firewall. You could have the firewall part without the translation part (or with deterministic two-way translation), which would eliminate 99% of problems. let me explain: A NAT "hides the network" behind it because it does two things: 1. drops incoming connections (or more precisely, incoming traffic when there was no outgoing traffic from that ip:port on that protocol) 2. obscures internal network addresses Dropping incoming connections is what makes it a simple firewall and you could set up a "real" firewall to do the same (there are probably products doing that out of the box). Most firewalls are more complex than that, but they don't _have to be_. What makes NATs different is the address translation that obscures the structure of the network behind it. Traditionally this is needed because you have lots of internal addresses and few external addresses (often just one). But if you have more hosts than external IPs then you can't have a static mapping without collisions because you have more internal ip:port pairs than external ip:port pairs. Which is why NAT generates internal-external mappings as it goes and why old mappings need to be discarded and why it needs to drop incoming connections (it can't know where to forwarded it to if there is no mapping). THIS is the part people hate, because it means you can't reliably connect two NATed hosts. If hole punching worked reliably it wouldn't be a problem, it feels like a hack, but the internet is a patchwork of much worse hacks.

peterwwillis 13y ago

http://nmap.org/misc/split-handshake.pdf http://nmap.org/misc/split-handshake.pdf

ricardobeat 13y ago

Isn't this called "NAT punching", and generally used by games and other sw to establish peer-to-peer connections?

DennisP 13y ago
That's usually UDP. With this method, you "get all of the utility [TCP] provides without trying to reproduce it over UDP."

chacham15 13y ago

From my understanding a NAT is going to remap an internal port to a different external port and upon receiving a RST will delete the mapping. Is this not correct? Also, doesnt connection reversal solve this problem? (Connection reversal: have A connect to intermediary S. have B connect to intermediary S. S sends A B's info and vice versa. A connects to B before closing connection with S.) This also does not require S to forward data, only connection information. Am I missing something there?

digitalsushi 13y ago
There are like 4 different NAT port translation models. In some of the models, the NAT algorithm will actually preserve the ephemeral port number in the NAT mapping. This falls apart quickly when you have a few NAT clients. But it can make legacy software work a lot better. We have weeded most of that software out at this point, or at least provided application gateways that inspect the content of the packet and tweak little things - consider how you need an extra kernel module for FTP and SIP and stuff. The models are detailed nicely in this diagram. https://en.wikipedia.org/wiki/Network_address_translation#Methods_of_port_translation https://en.wikipedia.org/wiki/Network_address_translation#Me...

latitude 13y ago

This works and it works reasonably well. I had used simultaneous-open TCP punching as a fallback for UDP punching when I was doing [0] and it did help in 10-20% of cases when (a fairly elaborate version of) UDP punching failed. That's on a scale of several hundred thousand mediated connections per day. One caveat though is that it requires implementing a bot-like functionality in the clients, meaning that a mediating server should be able to tell a client - "create a socket, bind it to this ip:port, wait, wait... connect to that ip:port". Obviously, this is an ideal platform for DDoS attacks if someone ever manages to re-point clients to a rogue mediation server. So, yeah, it works well, but there are some not so obvious trade-offs. [0] http://swapped.cc/hamachi http://swapped.cc/hamachi

steeve 13y ago
Ah! So that's how you guys did it! I always wondered what you were doing beyond UDP hole punching.
malloc2x 13y ago
Is there a Free software package that does something like this today?
- synctext 13y ago
  TCP simultaneous open is very difficult to use. In our experience with a deployed P2P system, UDP hole punching is more successful with all the strange NAT boxes deployed. Our success rate is 95.3% now in the wild. As said above, this TCP fallback sounds like an excellent way to get to 97% connection success rate! Note that there is an upcoming IETF Internet Standard describing UDP hole punching: http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-06#section-3.10 http://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-06#... We've implemented this as LGPL code: svn.tribler.org/libswift/branches/ppsp-03/

eridius 13y ago

Because nobody has linked it yet, the referenced XKCD is http://xkcd.com/949/ http://xkcd.com/949/

maffydub 13y ago

I'm surprised this works... or at least works reliably. I understood many simple firewalls just blocked inbound SYNs (without performing any kind of IP address lookup). This would obviously prevent this mechanism from working. Is this not (or no longer) the case?

amenod 13y ago
I am not an expert in this area, but from what I understand, firewalls keep a list of outbound connections. If inbound connection comes from a known destination IP+port, it will be forwarded to the internal "source" IP+port. This is how I understand it: if both A and B are behind firewalls, they use C to reach an agreement about IPs and ports used. Then A sends a packet to B, which is silently dropped at B's firewall. Then B send a packet to A - since it looks like an "answer" to previous request it is forwarded by A's firewall to A. Then A sends another packet to B, which is also forwarded by B's firewall to B. Voila, connection made. :) Note that this is just my understanding, so I would appreciate if someone more knowledgeable in this area would chime in.

wtracy 13y ago

Is this the sort of technology that P2P systems like BitTorrent use? I've never understood how BT works in setups that block incoming connections.

AndreasFrom 13y ago

BitTorrent Sync [1] can help with the explicit file sharing problem from the xkcd. Too bad it's not open source. [1]: http://labs.bittorrent.com/experiments/sync.html http://labs.bittorrent.com/experiments/sync.html

czzarr 13y ago

what was the title of the book you read?

throwawaykf02 13y ago

I'm not sure what OP has against UPnP port mapping. It's not an ideal option and certainly not the best-designed specification, but it's a way to explicitly tell the NAT to do what we want, rather than having to trick it into doing what we want. Also, one option not mentioned here yet is Teredo: http://en.m.wikipedia.org/wiki/Teredo_tunneling http://en.m.wikipedia.org/wiki/Teredo_tunneling It's primarily an IPv6 transition technology, but it comes with NAT traversal, hole-punching and all that built-in. What you get is a virtual IPv6 network interface to which you simply bind your socket, and you can connect to other Teredo / IPv6 sockets... if the whimsies of the Gods of Middleboxes allow. What makes it attractive is that it is deployed on all Windows machines WinXP and later (and enabled by default on Vista and later), giving it a huge deployment base. It's not present on non-Windows machines, of course, but there is a liberally licensed implementation for OSX and Linux called Miredo. uTorrent is one popular application that uses Teredo. However, Teredo tunneling does not work very reliably (apparently, its designers traded off connectivity for additional security), and it would be unadvisable to have that as your only NAT traversal method. But I think getting that option with minimal additional code is not a bad deal.