9 ms·

How NAT traversal works (2020)

305 points by punnerud 3y ago

seabass-labrax 3y ago
[2020]
- tadfisher 3y ago
  [NAT] was thinking, traversing Nat is dead-simple
- gjvc 3y ago
  recency troll
- yuvadam 3y ago
  And yet it's the bible of NAT traversal.
  - wmf 3y ago
    Nah, this is the bible of NAT traversal: https://bford.info/topics/Network-Address-Translation/ https://bford.info/topics/Network-Address-Translation/ (2005)
    - api 3y ago
      This old post too: https://www.zerotier.com/blog/the-state-of-nat-traversal/ https://www.zerotier.com/blog/the-state-of-nat-traversal/ There are lots of others. The TS post is well written though, and contains some interesting information like the way the birthday paradox relates to punching symmetric NATS.
  - dang 3y ago
    It's just the convention to put the year in the title when the article is older than a year or so. Historical material is always welcome on HN and it's fun for readers to know what year an article dates from.
  - commandlinefan 3y ago
    Man I wish there was more documentation like this out there, too.
- dang 3y ago
  Added. Thanks!
Jyaif 3y ago
> For example, we’ve observed that the UC Berkeley guest Wi-Fi blocks all outbound UDP except for DNS traffic. No amount of clever NAT tricks is going to get around the firewall eating your packets. I'm not familiar with DNS at all, but can't you craft UDP packets that look like DNS packets, but contain a useful payload with which you can do the whole STUN/UDP-hole-punching/p2p dance, assuming you have matching that can unwrap the payload?
- stevekemp 3y ago
  TCP over DNS has been a thing for a long time, though I guess it's fallen out of favour because "real" VPNs are so easily available nowadays. For example you can see here: https://analogbit.com/software/tcp-over-dns/ https://analogbit.com/software/tcp-over-dns/
- Hikikomori 3y ago
  Smarter firewalls can filter not just ports but the payload as well.
  - deleted 3y ago
    [deleted]
- gnfargbl 3y ago
  Even if you could, the problem would come when you wanted to establish a data connection with the remote end. Assuming "except for DNS traffic" actually means "except for packets on the well-known port for DNS" then you are going to be restricted to sending packets with destination port 53. This means that for comms to work, the remote end is going to have to be able to receive packets on port 53. Some kinds of network setups may allow this -- all of this stuff ends up being hideously dependent on the detail -- but many won't.
- cephei 3y ago
  It certainty depends on how the network is doing the packet scanning and the other measures in place. If the security measures include a allow list of DNS servers, then there's little that can be done to bypass it.
gnfargbl 3y ago
> (NATs do not enhance security in any meaningful way, but that’s a rant for another time.) Eh, I'm not sure I agree. People plug all kinds of weird and wonderful crap into their home networks, they turn off their Windows firewall and allow connections to RPC ports, all sorts of nonsense. The main factor that prevents even more stuff being exposed to the internet than now (and there's a lot exposed already) is the prevalence of IPv4 NAT devices. OK, UPnP is a thing, and NATs aren't firewalls, but in practice NATs do look and feel enough like a firewall to be at least somewhat useful to the average home user.
- johncolanduoni 3y ago
  Every router firewall in existence supports some sort of established/related only rule. That’s sufficient to provide all security benefits of a NAT except the hiding of which devices behind the router are making which connections.
  - noselasd 3y ago
    Only if those were on by default. Back in the days when my ISP didn’t do NAT , their dsl modem didn’t block anything. Most people do not have the capability to secure their internet connection. I still think the op has a good point in that NAT has protected many home networks , if nothing else from ISP incompetence , even if in theory a firewall could have done it better.
    - johncolanduoni 3y ago
      Modems don't generally block anything, unless they're a modem/router combo. Have you ever actually encountered a device that had a NAT but no stateful firewall?
    - samcat116 3y ago
      They are on by default now. Have been for years.
    - throw0101b 3y ago
      > Only if those were on by default. My Asus RT-AC68U, released in 2013, did/does (by default). * https://www.pcmag.com/reviews/asus-rt-ac68u-dual-band-wireless-ac1900-gigabit-router https://www.pcmag.com/reviews/asus-rt-ac68u-dual-band-wirele... Are you saying routers in 2023 do not? Can you list any examples of currently shipping CPEs that do not?
dang 3y ago
Related: How NAT traversal works (2020) - https://news.ycombinator.com/item?id=30707711 https://news.ycombinator.com/item?id=30707711 - March 2022 (37 comments) How NAT Traversal Works - https://news.ycombinator.com/item?id=24241105 https://news.ycombinator.com/item?id=24241105 - Aug 2020 (28 comments)
klabb3 3y ago
This is an incredible resource on NAT traversal in practice. I used it as a reference while building a server-assisted p2p-over-TCP[1] system. The post covers UDP but all of the theory applies to TCP as well – you just need some SO_REUSEPORT dual dial/listen magic so the code will look a bit different. Meta: The post alludes to this, but I think NAT traversal is an inaccurate and confusing term. I like “hole punching” a lot more as a general term. It gets the message across without the myopic connotation that it’s all about NATs – it isn’t. [1]: https://github.com/betamos/rdv https://github.com/betamos/rdv
JoeAltmaier 3y ago
It's a good primer. It's also more complicated than that. NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. Port number can be as important as IP address. The STUN server can see one port number, but a different one may be used by your gateway for talking to the real desired endpoint. Result: fail, at least for TCP I used to do all this but used UDP. One port for talking to everybody. Had to tag packets with some kind of UID so I knew who it was from, since every conversation came thru the same port. That worked better. Still UDP is the red-headed stepchild of the internet. Some countries(!) didn't used to allow it. Some corporations don't. Some routers screw it up regularly - drop packets, reorder packets, don't do reassembly if they get fragmented. Had to recover from all that. My approach was to use WiFi size packets (<=512 bytes) with a UID and a single port for everything. Put sequence numbers in them, and reorder if they got swapped (a regular thing with some routers). And always be prepared to revert to TURN (your public server that forwards packets for you).
- klabb3 3y ago
  > It's a good primer. It's also more complicated than that. Subjective, but I think primer is an understatement. Tailscale did an absolute stellar job of simplifying the problem to something that works very well in practice, and is relatively simple to implement. > NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. Yes, this is a big issue and indeed a source of a lot of stateful complicated logic in the tailscale client and anyone else who attempts it. And that’s aside from the absolute clusterfuck that is UPnP – super complicated for what it does. In my view it’s not enough bang for the buck; if STUN won’t work (some mobile carriers and enterprise networks) then port-mapping protocols probably won’t work either. You gotta cut your losses somewhere. > The STUN server can see one port number, but a different one may be used by your gateway for talking to the real desired endpoint. The post covers this in detail. They call it endpoint-dependent mapping.
  - anyfoo 3y ago
    > > NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. > Yes, this is a big issue [...] You gotta cut your losses somewhere. And the article also gets into that, in detail. It appears that tailscale not only dynamically retries, they also test out all possible paths to switch to the best one.
teleforce 3y ago
NAT, the classic example of technical debt created when scalability is not in every designer mind.
MzHN 3y ago
The article goes through all of this, but since it starts with > For example, WebRTC uses this bag of tricks to send peer-to-peer audio, video and data between web browsers. I think it is worth emphasizing that since web users are increasingly mobile users, having P2P on the web is nearly impossible. Mobile users have the worst kinds of CGNATs with everything randomized for every connection. Throw in a couple of IPv4 only non-mobile clients, and your IPv6 mobile users can't even connect to them even if they didn't have NAT. So for my use cases the TL;DR of the article is: "How NAT traversal works", it does not. At least not on the web. Which is a shame.
- dave_universetf 3y ago
  That is not accurate to our experience. Mobile devices on cell networks see about the same amount of success with p2p connections as other devices. Mobile networks differ vastly in their behaviors when it comes to NAT traversal, and you can certainly find specific carriers that behave poorly, but it's far from the death of p2p that you describe. That said, you _do_ need last-resort relaying, not just for the more difficult mobile networks but for all kinds of edge cases (e.g. the UC Berkeley guest wifi case I mentioned in the article). So in that sense, the purist argument is that p2p is impossible since you will always need to have relaying as an option. But the key is that p2p works well enough to make the relays economically viable to operate, and that _some_ connectivity is usually sufficient, even if it's not the theoretically optimal connectivity. Plus, IPv6 continues to be deployed (and AWS charging for IPv4 is another step on that path), so while some CGNAT deployments work to degrade p2p, IPv6 in parallel is restoring better p2p connectivity than IPv4's allowed for decades. It's not all roses, but it's possible to be optimistic about a future where cloud servers aren't the only option.
  - MzHN 3y ago
    Thanks for the optimism. I guess I'm just salty, that 100% of my users can't connect to 100% of my users, but that is only because I'm building highly local hobby projects, so I can't see the global picture where the % is a lot more sensible.

305 points by punnerud 3y ago

seabass-labrax 3y ago

[2020]

tadfisher 3y ago
[NAT] was thinking, traversing Nat is dead-simple
gjvc 3y ago
recency troll
yuvadam 3y ago
And yet it's the bible of NAT traversal.
- wmf 3y ago
  Nah, this is the bible of NAT traversal: https://bford.info/topics/Network-Address-Translation/ https://bford.info/topics/Network-Address-Translation/ (2005)
  - api 3y ago
    This old post too: https://www.zerotier.com/blog/the-state-of-nat-traversal/ https://www.zerotier.com/blog/the-state-of-nat-traversal/ There are lots of others. The TS post is well written though, and contains some interesting information like the way the birthday paradox relates to punching symmetric NATS.
- dang 3y ago
  It's just the convention to put the year in the title when the article is older than a year or so. Historical material is always welcome on HN and it's fun for readers to know what year an article dates from.
- commandlinefan 3y ago
  Man I wish there was more documentation like this out there, too.
dang 3y ago
Added. Thanks!

Jyaif 3y ago

> For example, we’ve observed that the UC Berkeley guest Wi-Fi blocks all outbound UDP except for DNS traffic. No amount of clever NAT tricks is going to get around the firewall eating your packets. I'm not familiar with DNS at all, but can't you craft UDP packets that look like DNS packets, but contain a useful payload with which you can do the whole STUN/UDP-hole-punching/p2p dance, assuming you have matching that can unwrap the payload?

stevekemp 3y ago
TCP over DNS has been a thing for a long time, though I guess it's fallen out of favour because "real" VPNs are so easily available nowadays. For example you can see here: https://analogbit.com/software/tcp-over-dns/ https://analogbit.com/software/tcp-over-dns/
Hikikomori 3y ago
Smarter firewalls can filter not just ports but the payload as well.
- deleted 3y ago
  [deleted]
gnfargbl 3y ago
Even if you could, the problem would come when you wanted to establish a data connection with the remote end. Assuming "except for DNS traffic" actually means "except for packets on the well-known port for DNS" then you are going to be restricted to sending packets with destination port 53. This means that for comms to work, the remote end is going to have to be able to receive packets on port 53. Some kinds of network setups may allow this -- all of this stuff ends up being hideously dependent on the detail -- but many won't.
cephei 3y ago
It certainty depends on how the network is doing the packet scanning and the other measures in place. If the security measures include a allow list of DNS servers, then there's little that can be done to bypass it.

gnfargbl 3y ago

> (NATs do not enhance security in any meaningful way, but that’s a rant for another time.) Eh, I'm not sure I agree. People plug all kinds of weird and wonderful crap into their home networks, they turn off their Windows firewall and allow connections to RPC ports, all sorts of nonsense. The main factor that prevents even more stuff being exposed to the internet than now (and there's a lot exposed already) is the prevalence of IPv4 NAT devices. OK, UPnP is a thing, and NATs aren't firewalls, but in practice NATs do look and feel enough like a firewall to be at least somewhat useful to the average home user.

johncolanduoni 3y ago
Every router firewall in existence supports some sort of established/related only rule. That’s sufficient to provide all security benefits of a NAT except the hiding of which devices behind the router are making which connections.
- noselasd 3y ago
  Only if those were on by default. Back in the days when my ISP didn’t do NAT , their dsl modem didn’t block anything. Most people do not have the capability to secure their internet connection. I still think the op has a good point in that NAT has protected many home networks , if nothing else from ISP incompetence , even if in theory a firewall could have done it better.
  - johncolanduoni 3y ago
    Modems don't generally block anything, unless they're a modem/router combo. Have you ever actually encountered a device that had a NAT but no stateful firewall?
  - samcat116 3y ago
    They are on by default now. Have been for years.
  - throw0101b 3y ago
    > Only if those were on by default. My Asus RT-AC68U, released in 2013, did/does (by default). * https://www.pcmag.com/reviews/asus-rt-ac68u-dual-band-wireless-ac1900-gigabit-router https://www.pcmag.com/reviews/asus-rt-ac68u-dual-band-wirele... Are you saying routers in 2023 do not? Can you list any examples of currently shipping CPEs that do not?

dang 3y ago

Related: How NAT traversal works (2020) - https://news.ycombinator.com/item?id=30707711 https://news.ycombinator.com/item?id=30707711 - March 2022 (37 comments) How NAT Traversal Works - https://news.ycombinator.com/item?id=24241105 https://news.ycombinator.com/item?id=24241105 - Aug 2020 (28 comments)

klabb3 3y ago

This is an incredible resource on NAT traversal in practice. I used it as a reference while building a server-assisted p2p-over-TCP[1] system. The post covers UDP but all of the theory applies to TCP as well – you just need some SO_REUSEPORT dual dial/listen magic so the code will look a bit different. Meta: The post alludes to this, but I think NAT traversal is an inaccurate and confusing term. I like “hole punching” a lot more as a general term. It gets the message across without the myopic connotation that it’s all about NATs – it isn’t. [1]: https://github.com/betamos/rdv https://github.com/betamos/rdv

JoeAltmaier 3y ago

It's a good primer. It's also more complicated than that. NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. Port number can be as important as IP address. The STUN server can see one port number, but a different one may be used by your gateway for talking to the real desired endpoint. Result: fail, at least for TCP I used to do all this but used UDP. One port for talking to everybody. Had to tag packets with some kind of UID so I knew who it was from, since every conversation came thru the same port. That worked better. Still UDP is the red-headed stepchild of the internet. Some countries(!) didn't used to allow it. Some corporations don't. Some routers screw it up regularly - drop packets, reorder packets, don't do reassembly if they get fragmented. Had to recover from all that. My approach was to use WiFi size packets (<=512 bytes) with a UID and a single port for everything. Put sequence numbers in them, and reorder if they got swapped (a regular thing with some routers). And always be prepared to revert to TURN (your public server that forwards packets for you).

klabb3 3y ago
> It's a good primer. It's also more complicated than that. Subjective, but I think primer is an understatement. Tailscale did an absolute stellar job of simplifying the problem to something that works very well in practice, and is relatively simple to implement. > NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. Yes, this is a big issue and indeed a source of a lot of stateful complicated logic in the tailscale client and anyone else who attempts it. And that’s aside from the absolute clusterfuck that is UPnP – super complicated for what it does. In my view it’s not enough bang for the buck; if STUN won’t work (some mobile carriers and enterprise networks) then port-mapping protocols probably won’t work either. You gotta cut your losses somewhere. > The STUN server can see one port number, but a different one may be used by your gateway for talking to the real desired endpoint. The post covers this in detail. They call it endpoint-dependent mapping.
- anyfoo 3y ago
  > > NAT mappings can change (your router can reboot etc). So you have to be ready to 'try again' dynamically. > Yes, this is a big issue [...] You gotta cut your losses somewhere. And the article also gets into that, in detail. It appears that tailscale not only dynamically retries, they also test out all possible paths to switch to the best one.

teleforce 3y ago

NAT, the classic example of technical debt created when scalability is not in every designer mind.

MzHN 3y ago

The article goes through all of this, but since it starts with > For example, WebRTC uses this bag of tricks to send peer-to-peer audio, video and data between web browsers. I think it is worth emphasizing that since web users are increasingly mobile users, having P2P on the web is nearly impossible. Mobile users have the worst kinds of CGNATs with everything randomized for every connection. Throw in a couple of IPv4 only non-mobile clients, and your IPv6 mobile users can't even connect to them even if they didn't have NAT. So for my use cases the TL;DR of the article is: "How NAT traversal works", it does not. At least not on the web. Which is a shame.

dave_universetf 3y ago
That is not accurate to our experience. Mobile devices on cell networks see about the same amount of success with p2p connections as other devices. Mobile networks differ vastly in their behaviors when it comes to NAT traversal, and you can certainly find specific carriers that behave poorly, but it's far from the death of p2p that you describe. That said, you _do_ need last-resort relaying, not just for the more difficult mobile networks but for all kinds of edge cases (e.g. the UC Berkeley guest wifi case I mentioned in the article). So in that sense, the purist argument is that p2p is impossible since you will always need to have relaying as an option. But the key is that p2p works well enough to make the relays economically viable to operate, and that _some_ connectivity is usually sufficient, even if it's not the theoretically optimal connectivity. Plus, IPv6 continues to be deployed (and AWS charging for IPv4 is another step on that path), so while some CGNAT deployments work to degrade p2p, IPv6 in parallel is restoring better p2p connectivity than IPv4's allowed for decades. It's not all roses, but it's possible to be optimistic about a future where cloud servers aren't the only option.
- MzHN 3y ago
  Thanks for the optimism. I guess I'm just salty, that 100% of my users can't connect to 100% of my users, but that is only because I'm building highly local hobby projects, so I can't see the global picture where the % is a lot more sensible.