8 ms·
WireGuard Bounce Server Setup
- jjice 6y agoSeems like a good tutorial. I've been using this type of set up for about 6 months now and it works well for me. I wish I had a straight forward explanation like this when I was first learning how to use WireGuard.
- momothereal 6y agoSame! I set up something similar with DigitalOcean by combining a couple of tutorials together. I pretty much followed the same steps until the `nftables` stuff.
- mbreese 6y agoThat was my experience setting up a WireGuard connection to my home network about 12 months ago. It was really difficult to get a straightforward explanation of how to set things up. For example, I had (as mentioned above) dynamic DNS set to update from my home router with port forwarding setup to a home Linux server (Rpi actually). This isn't as easy as A <=> B, but it is probably a common enough use-case. It was really hard to figure out what I needed to put in my wg.conf files. But once I got it, it was really solid. Is there something like "WireGuard in anger" or even something like the GitHub introduction to Git for WireGuard? It seems like it would be very helpful.
- ahnick 6y agoI must be missing something. Why go through all of this when you can just buy a domain name, setup dynamic dns on the natted network (to keep the IP updated), and then setup wireguard to route to the domain name? If you have two different networks then just use two different domains or subdomains.
- ncmncm 6y ago(Author here, AMA.) If that does what you need, then it is a reasonable alternative. I would prefer the shorter routing you get. But if one of your NATs is corporate, a phone carrier, it might not work. If you might want to provide other services, particularly a remote exit node or web service, the bounce node is a good start on that.
- kuroguro 6y agoWould that setup punch trough the nat tho?
- ahnick 6y agoYes, you setup the wireguard configuration to connect to the peer using the domain name and once connected, the network assigns a local IP for the connected peer that is valid on the natted network. I use this kind of setup to access my NAS system when I am traveling or working away from home.
- mbreese 6y agoYou don't need to punch through a NAT when you have a bounce server. Both sides are already connected to the bounce server and all interaction between those peers is done through the bounce server. Thus, the peers don't need to communicate directly... so you don't need to do any NAT punching. That's what makes this method simple... with the downside that you need a publicly exposed intermediary. But it is vastly simpler to setup than STUN.
- stvsu 6y agoOPs method would allow for remoting into your home network without port forwarding, since home network would establish a connection to the 'bounce' node, which would facilitate communication between the 3rd WG client
- ahnick 6y agoTrue, but on the bounce server it looks like the ports are just being managed via nftables. Isn't that just a wash?
- ncmncm 6y agoDon't see how... Nftables is set up once and then left alone. Or could as well just be left alone entirely. For minimal setup, zerotier looks like the winner.
- ahnick 6y ago> Don't see how... Nftables is set up once and then left alone. Same thing for a router configuration to accept inbound wireguard requests on a home network with dyndns. You set it up once and are done. I see the benefit of the bounce server if you operate a network in an environment where you don't have the ability to control the router config; however, when you do have the ability to update firewall/router config, then I'd prefer just setting up a domain name and avoid the dependency on a third party server.
- ncmncm 6y agoDyndns does work when you have a say in at least one side of each potential connection, unless you want one of your NATted hosts to also act as a bounce server in a pinch. But you do not seem to be getting that the use of nftables, for the open-network bounce server, is wholly optional.
- ahnick 6y agoah, okay. Yes, I missed that. So the point of nftables then is just to avoid sending REJECT messages, so it makes it harder to determine what port wireguard is operating on?
- aftbit 6y agoWhat if you live in a college dorm or use apartment wifi where they do not give you control of the port forwarding rules? Or what if you want to deploy devices into other peoples' networks (e.g. IoT)? What if you're sending someone a bootable USB image so you can do data recovery on their hard drive in situ without them having to ship it to you? What if you're behind a carrier-grade NAT and don't have a public IPv4 address at all? In these cases and many more, you might find a wireguard bounce host useful.
- ahnick 6y agoHonestly, I'd probably just use Tailscale for that, so I guess the bounce server solution occupies some middle ground where one doesn't have control over the network and either doesn't have the money for tailscale or likes owning their infra end-to-end.
- aftbit 6y agoFor me, it's indeed about FLOSS and owning the system end-to-end. Plus, I want to be able to build things like Tailscale, and I can't very well do that just using the existing tools. :)
- yokem55 6y agoI was hoping this was some kind of stun mechanism for wireguard. No, it's just setting up a cheap wireguard server on AWS. Anyone have anything like an introduction server to help wg peers behind nat find each other?
- btgeekboy 6y agoThat’s kinda what Tailscale does, amongst other things.
- mr-wendel 6y agoYup, it's definitely one of the things Tailscale does. Not an intro or tutorial, but I've not found a better write-up than on their blog: https://tailscale.com/blog/how-nat-traversal-works/ https://tailscale.com/blog/how-nat-traversal-works/ (personally, I'd just do what the author did... it's a ton less work to setup and maintain)
- aftbit 6y agoThis[1] is the best post I've seen on that _outside_ of the Tailscale writeup linked below. The clever bit is that the introduction server uses wireguard tunnels to find your endpoint information, then shares it out via DNS. Of course it still requires you to be able to run custom code on all of the endpoints, which requires supporting many different platforms. I'm still looking for a FLOSS mesh network built on top of wireguard that can do NAT traversal between nodes and fall back to tunnelling traffic for the annoying cases where this fails. I don't really want to use Tailscale because (1) their server is not open (though this should not matter a ton for trust) and (2) they require a Google account or similar to sign up. 1: https://www.jordanwhited.com/posts/wireguard-endpoint-discovery-nat-traversal/ https://www.jordanwhited.com/posts/wireguard-endpoint-discov...
- mbreese 6y agoI can't see how the post you linked actually does the NAT traversal... Alice finds Bob's external IP:port using the registry. That makes sense. But doesn't Bob need to send a packet to Alice to setup the NAT traversal on his side? More accurately, Alice uses the SRV field to populate the wg peer information on her side -- but how does Bob know that he needs to update the peer information on his side? I think this is really close, but maybe using a custom DNS server for this might be trying to be a little too clever?
- gruez 6y agoThis works if both sides has wireguard installed and are in the network, but is there a way to got this to work for any connections from the internet? eg. your VPS has ip 1.1.1.1, you want any traffic that goes to 1.1.1.1 to go to the wireguard interface on your PC, and any traffic set from the wireguard interface on your PC exits through 1.1.1.1. Bonus points if set it up in such so that the interface address on your PC shows up as 1.1.1.1.
- aftbit 6y agoYes this is relatively straightforward to accomplish via iptables pre&postrouting rules. I think the key insight is to realize that you're basically asking for NAT with DMZ: Internet -> [eth0] WG BOX [wg0] -> [wg0] HOME SERVER analogous to: Internet -> [eth0] ROUTER [eth1] -> [eth0] HOME SERVER
- zokier 6y agoYou probably want to have two (or more) interfaces (eni in aws lingo) on the wg server so that you dedicate one for this tunneled traffic. Also you probably want to setup some sort of NAT instead of just routing packets directly.
- zokier 6y agoThat was kinda the article I was expecting to read, so I was very confused when it didn't actually go there.
- 1vuio0pswjnm7 6y ago"If you have someplace to put more secure equipment on the open net, that would be better. A [your choice of HW] would do fine, if it runs only your code." Idea: "Colocation centres" for users' computers instead of data centres for users' data. As directed by the author here, users would store no data on these computers.1 1. By their nature, each user-owned supernode computer would provide some discoverable metadata, as would any router, namely, the IP addresses of the members of the private network/s it supports and which of those IPs are connecting to each other, but this would be unlike the centralised repositories of millions of users' metadata we have now, managed by private tech companies. As for the rest of this writeup, it sounds much like the need for a supernode in "LAN-over-Internet"-type P2P networks that encapsulate Ethernet-like packets in UDP packets. The supernode does not necesarily need to route traffic between nodes behind NAT (rarely necessary), it only needs to store the equivalent of an ARP table accessible by all of them.
- gruez 6y ago>Idea: Colocation centres for users' computers instead of data centres for users' data. You mean this? https://en.wikipedia.org/wiki/Colocation_centre https://en.wikipedia.org/wiki/Colocation_centre
- 0xbadcafebee 6y ago> Idea: "Colocation centres" for users' computers instead of data centres for users' data. As directed by the author here, users would store no data on these computers. This is basically AWS Workspaces. It can work, but it's annoying, expensive, and various latency and bandwidth issues are a killer.
- 1vuio0pswjnm7 6y agoLove how they market that: "Desktop as a Service". Contrast with tiny, self-served routers owned by users that store no user data and whose only access is via SSH. Who owns the computers in the "AWS Workspace". Need to see the terms the AWS customer must agree to (non-negotiable, no doubt) to get a better understanding of what this service is really about.
- _hyn3 6y agohttps://zerotier.com https://zerotier.com does a very nice job of NAT "bouncing", free (hosted) for up to 50 hosts, or you can run your own. You end up with another "zt" interface, so you can apply firewall rules to that interface, separate from your main NIC(s), kinda like assigning your own VPC but for your laptop, etc. It only takes a few seconds to install the agent and join your private network, and it seems to reconnect quite nicely as well; if your network is private, you can force all new attempts to connect to be approved (where you can check the mac address first). Basically, it's exactly the right level of technical config traits with a simple interface. (no affiliation, just a very happy user, but I will say that running your own seems to be a lot more difficult than it should be..)
- aftbit 6y agoIs zerotier FLOSS? I have been looking for something to fulfill this need, but especially after the Solarwinds hack, I am unwilling to run a closed-source agent on any of my hosts.
- evil-olive 6y agoLooks like the Business Source License [0]. Which does not qualify under the Open Source Definition [1], but they claim it meets "most" of the OSI criteria. 0: https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.txt https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.... 1: https://mariadb.com/bsl-faq-adopting/#osl https://mariadb.com/bsl-faq-adopting/#osl
- jlgaddis 6y ago> ... but they claim it meets "most" of the OSI criteria. As Frank Robinson [0] once said: > "[...] Close only counts in horseshoes [1] and hand grenades." -- [0]: https://en.wikipedia.org/wiki/Frank_Robinson https://en.wikipedia.org/wiki/Frank_Robinson [1]: https://en.wikipedia.org/wiki/Horseshoes https://en.wikipedia.org/wiki/Horseshoes (for the uninitiated)
- 6y ago
- aftbit 6y agoI've had basically this exact setup running on a cloud server for the past few months. It's pretty liberating to be able to open ports to 10.44.0.0/24 and have all of my different machines easily access them, regardless of what networks they're connected to. I've added in an instance of nsd that manages a zonefile so I can do: ssh laptop.wg.mydomain.net which resolves into 10.44.0.3 for example. There are two major downsides with this approach: (1) all of your traffic must go by way of the bounce server, even if your machines are on the same LAN. (2) the bounce host can observe all of your traffic. I've been considering adding a second layer of wireguard on top of this for end-to-end encryption between nodes. Each machine would have one interface configured like this article suggests simply to assign a stable NAT-traversing IP address. Then they'd each have a second interface using the first-level IP addresses as endpoints. This would result in 2x the encryption overhead, both in CPU and more importantly in MTU, but it would mean the bounce host was no longer able to intercept even unencrypted data. If you run a daemon on all of the hosts, they can play with the endpoints used on this second level network to avoid using the bounce host when they find they can directly connect to each other, but the default path would work (if somewhat slowly) by the double tunnel process even without this daemon. For bonus points, run multiple bounce hosts, connect to all of them from each peer, then select the endpoint to use for each paired peer connection based on the total path latency.
- ncmncm 6y agoI was thinking I would need to do a Wg-in-wg tunnel layer until I realized all my traffic is already tunneling via ssh or ssl anyway.
- jlgaddis 6y agoI'm not certain I grok all of that correctly, but the last bit sounds kinda like DMVPN [0]. -- [0]: https://en.wikipedia.org/wiki/Dynamic_Multipoint_Virtual_Private_Network https://en.wikipedia.org/wiki/Dynamic_Multipoint_Virtual_Pri...
- lights0123 6y agoDoesn't Ubuntu 20.04 have a kernel with Wireguard built in? You shouldn't need to use DKMS.
- aftbit 6y agoEven on my 20.04 box, I seem to have wireguard-dkms auto-installed in response to "apt install wireguard". Perhaps they are doing this to allow for faster wireguard module updates? Or perhaps this is just a bug. https://launchpad.net/bugs/1890201 https://launchpad.net/bugs/1890201 https://launchpad.net/ubuntu/+source/wireguard/1.0.20200513-1~20.04.2 https://launchpad.net/ubuntu/+source/wireguard/1.0.20200513-...
- magicconch 6y agoUsing “apt install --no-recommends wireguard-tools” should do what you’re looking for.
- ncmncm 6y agoBelt-and-suspenders... I assumed some readers might choose a different distro that needs a little more setup.
- lights0123 6y agoI'm looking at doing something similar: does the server actually need an IP address assigned to its wg0 interface, or can it work without one? I only need peer-to-peer.
- aftbit 6y agoYes, because of Cryptokey routing[1]. Wireguard works at the IP level and determines which keys to use for encryption and authentication based on the destination IP. That said, if you only need peer-to-peer, you could pick any two RFC-1918 addresses like 10.0.0.1 and 10.0.0.2. 1. https://www.wireguard.com/#cryptokey-routing https://www.wireguard.com/#cryptokey-routing
- IncludeSecurity 6y agoThis is a great FOSS tool if you don't want to deal with all of the low level stuff. https://github.com/StreisandEffect/streisand https://github.com/StreisandEffect/streisand We did an audit of it and they fixed lots of configuration problems, it's now pretty solid security defaults wise. And the WG integration works well. They support the following setups: OpenSSH Tinyproxy OpenConnect / Cisco AnyConnect Stunnel Shadowsocks Obfsproxy WireGuard, Running on: Amazon Web Services (AWS) Microsoft Azure Digital Ocean Google Compute Engine (GCE) Linode Rackspace
- tptacek 6y agoThe generally-better-regarded implementation of this idea is Algo: https://github.com/trailofbits/algo https://github.com/trailofbits/algo
- anderspitman 6y agoIf you're interested in tunneling in general (such as ngrok and other ways to get through to private networks), I've been maintaining a list here: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling Always on the lookup for tools I may have missed.
- iampims 6y agohttps://github.com/cloudflare/cloudflared https://github.com/cloudflare/cloudflared would be one I didn’t see on your list.
- naggie 6y agoNice article. I had been maintaining a few servers doing a similar thing, but found the peer management, address allocation and key creation to be a bit tedious after some time so I created a command, https://github.com/naggie/dsnet/ https://github.com/naggie/dsnet/ -- adds peers in one command with IPv6 and PSK support.
- stock_toaster 6y agoA self hosted "TURN like" server seems like it would be simpler and more secure -- yet achieve similar goals, and yet still more robust vs funky NAT layers than a dynamic dns solution alone would provide.
- aftbit 6y agoI'd love to see a nice guide for setting this up. I've found the world of STUN/TURN/ICE incredibly confusing, probably because of the SIP/telephony background.
- pinusc 6y agoSomeone linked a write-up above [1]. It's not simple, but I found it well-written, and doesn't assume too much networking knowledge. I guess it's still a young protocol and we have proofs of concepts but not full-fledged deployment-ready solutions yet... 1: https://www.jordanwhited.com/posts/wireguard-endpoint-discovery-nat-traversal https://www.jordanwhited.com/posts/wireguard-endpoint-discov...
- aftbit 6y agoThat was me :)
- quantumofalpha 6y ago> To connect between NATted hosts, you need control of a host that is not Um, no? https://github.com/samyk/pwnat https://github.com/samyk/pwnat
- ohazi 6y agoSo we've known about this for years, but does anyone actually rely on this behavior in production, or are we all still pretending that this clever hack could be patched out of existence any day now? It clearly works. It's worked for a long time. But the common opinion still seems to be "well, this isn't supposed to work, so using it is a bit dodgy..."
- octoberfranklin 6y agoI don't think anybody sane tries to get two machines behind different NATs to talk to each other without the NATs' cooperation in production.
- quantumofalpha 6y agoWell, you need just one NAT to be cooperative, like a full cone. There's enough legitimate interest in NAT traversal that big NAT deployments like CGNATs tend to cooperate Traversing non-cooperative fully symmetric NATs, which randomize ports, is hard enough also for pwnat. Though in theory should be doable - you just need a lot of patience to brute force ports (there's only 64k of them) until it finally clicks
- ncmncm 6y agoThank you for this. I will try it.
- ncmncm 6y agoI tried it... SEGFAULT in the client: Null pointer to a port-number arg. I supplied all of the optional arguments...
- 6y ago
- jacob019 6y agoI use wireguard on a cheap VPS that comes with an IPV6 /64. Connecting devices get a public IPV6 address. Most of which are assigned a subdomain so I can remember them. It works wonderfully.
- artificialLimbs 6y agoDo you have a writeup or know where one is that could be of use in setting this up? I don't know much about v6...
- jacob019 6y agoNo, I just cobbled it together with standard Linux routing tools. echo 1 > /proc/sys/net/ipv6/conf/all/forwarding The trickiest part is making sure that the VPS answers NDP requests for the routed addresses: echo 1 > /proc/sys/net/ipv6/conf/all/proxy_ndp for i in `seq 0x0010 0x001f`; do ip=2001:1111:2222:3333::$(printf '%x' $i) ip neigh add proxy $ip dev ens3 done Assign the routed address range to the wireguard interface: ip addr add 2001:1111:2222:3333::10/124 dev wg0 That's the gist of it. I might do a proper writeup next week if I can find time.
- ncmncm 6y agoIf you do, please link it here.
- jlgaddis 6y ago> NOTE: Systemd has a nasty habit of delaying logins for 25 - 100 seconds while it waits for some service you never asked for to time out and fail, before it lets you have a prompt. If you experience this when logging in to a host via SSH, the delays are almost certainly due to either missing or non-functional forward/reverse DNS (the SSH server will perform these lookups when connections are received). You've got at least three options (in order of preference): - Add the appropriate (A/PTR) resource records to DNS so that the queries get a valid response - Add entries to /etc/hosts file - Set "UseDNS no" in the /etc/ssh/sshd_config file and add "-u 0" to the parameters passed to `sshd` when it's started (how to do this will vary by distribution/operating system)
- ncmncm 6y agoThank you. I haven't seen any apparent DNS problems; I fixed it, on a Debian host, by deleting the gnupg packages; probably deleting just gpgconf would have sufficed. That said, the IP addresses I used really would not show up in a DNS lookup. If I see it again, I will look for DNS lookup attempts. Dec 15 22:23:14 ip-99-99-99-100 sshd[1995]: pam_unix(sshd:session): session opened for user admin by (uid=0) Dec 15 22:23:14 ip-99-99-99-100 systemd[1]: Created slice User Slice of UID 1000. Dec 15 22:23:14 ip-99-99-99-100 systemd[1]: Starting User Runtime Directory /run/user/1000... Dec 15 22:23:14 ip-99-99-99-100 systemd-logind[510]: New session 34 of user admin. Dec 15 22:23:14 ip-99-99-99-100 systemd[1]: Finished User Runtime Directory /run/user/1000. Dec 15 22:23:14 ip-99-99-99-100 systemd[1]: Starting User Manager for UID 1000... Dec 15 22:23:14 ip-99-99-99-100 systemd[2001]: pam_unix(systemd-user:session): session opened for user admin by (uid=0) Dec 15 22:24:44 ip-99-99-99-100 systemd[2003]: pam_unix(systemd-user:session): session closed for user admin Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: user@1000.service: Main process exited, code=exited, status=1/FAILURE Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: user@1000.service: Killing process 2007 (gpgconf) with signal SIGKILL. Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: user@1000.service: Killing process 2008 (awk) with signal SIGKILL. Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: user@1000.service: Killing process 2013 (dirmngr) with signal SIGKILL. Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: user@1000.service: Failed with result 'exit-code'. Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: Failed to start User Manager for UID 1000. Dec 15 22:24:44 ip-99-99-99-100 systemd[1]: Started Session 34 of user admin. But what are these evidently unnecessary session and User Manager things? What controls starting them? What are they supposed to do for me, if they ever work right? Why did starting them fail?
- waynecrescent 6y ago>because Amazon techs and motivated hackers can peek at files you put there (yes, really, all of them! including your secret keys) What did the author mean by this? I can understand amazon employees having the ability to peek at private data but what about "hackers"? is he referring to the fact that lightsail is containerized and not running on VMs?(just a guess)