17 ms·

Putting Raspberry Pi Online with Caddy and SSH Tunnel

146 points by mathnmusic 6y ago

iamd3vil 6y ago
I think a better option would be to create a Wireguard tunnel between Raspberry Pi and the remote server instead of a SSH Tunnel. Then there is no need to add or change ports and restart the tunnel for every service.
- thespoonbends 6y ago
  OP's solution sound good enough for their needs. Solutions can always be improved, but it's not always worth doing that.
  - iamd3vil 6y ago
    OP has literally written in the gist about exploring a way to map entire port range and avoiding doing this, so the non hacky way of doing this is setting up something like a wireguard tunnel. That's the reason I suggested doing this instead of a tunnel which has other disadvantages like doing TCP on TCP.
    - sneak 6y ago
      SSH tunnels do not run TCP inside of them, just the bytes of the connection data itself. The only TCP in use is the TCP connection of the SSH connection between hosts.
      - iamd3vil 6y ago
        Ohh TIL but my other point still stands.
      - xorcist 6y ago
        It used to be common (at least not unhead of) to run ppp over ssh, which has this problem.
- mercora 6y ago
  while i think this is true (any other vpn software would work too though) i want to point out that you actually can bring up a tun interface using ssh with the "-w local_tun[:remote_tun]" flag somewhat easily if you want to. It is also possible to make forwarding work in either direction using an integrated socks proxy using "-R" or "-D" flags ...
  - anderspitman 6y ago
    > you actually can bring up a tun interface using ssh Beware of TCP over TCP issues[0] when using SSH for tun. [0]: http://sites.inka.de/bigred/devel/tcp-tcp.html http://sites.inka.de/bigred/devel/tcp-tcp.html
sigmaprimus 6y ago
Is your plan to run some sort of IOT application on the PI? I'm a little confused as to why you would go through the trouble of running a reverse proxy on a remote server that is fully capable of hosting your web applications and or just setting up a port forwarding rule on your local wifi router to redirect 10080 to 80.
- thespoonbends 6y ago
  I do a similar thing so I don't have to store sensitive data on a VPS. The sensitive data is used to compute non-sensitive results that I'm happy to go via a VPS.
- 3np 6y ago
  Privacy; keep data local, without exposing your residential IP.
  - BHSPitMonkey 6y ago
    Is exposing one's residential IP address such a bad thing? Is the threat model just that someone might wish to target you individually, and that they'll have better luck finding a vuln into your LAN if your services are listening directly on public ports?
    - 3np 6y ago
      There are unlimited imaginable reasons why one may want to de-correlate the data sent in or out from ones software to ones personal identity, yes.
      - BHSPitMonkey 6y ago
        I'd think that using a remote VPN is a better/broader solution to that particular problem. People don't often self-host things at the home, so it's unlikely anyone out there would see my DNS record and assume that it's of value and a useful tool for correlating my IP with my identity (and this actor would need to have eyes on a lot of different services' server logs in order to have data to _maybe_ correlate with me in the first place).
lhoff 6y ago
I have a similar setup but instead of SSH Tunnels i use wireguard. In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.
- allthingsequal 6y ago
  Could you please elaborate on your setup? I haven't had the opportunity to play with Wireguard yet, that could be a useful first project to me :-)
  - hakeberio 6y ago
    Pivpn is beginner-friendly and you can always branch off from there. https://www.pivpn.io https://www.pivpn.io
  - vsajip 6y ago
    I've started playing around with Tailscale [1], which is based on WireGuard. Free to use for a single user, and I've tried it out on Raspberry Pi, Linux x86_64, Windows and Android. Seems to work well, except there appears to be some issue with accessing Windows via the secure IP (neither ping nor Remote Desktop appear to get through the Windows firewall, despite having a specific firewall rule for Tailscale) [2]. Can SSH into e.g. Linux hosts using PuTTY from Windows without issues. [1] https://tailscale.com/ https://tailscale.com/ [2] https://github.com/tailscale/tailscale/issues/854 https://github.com/tailscale/tailscale/issues/854
  - 8K832d7tNmiQ 6y ago
    the idea is that your hetzner server acts as a gateway and you can communicate with your pi through the virtual private network, therefore removing any need to access through public ip.
- twicetwice 6y ago
  I did the same. It works really well for hole-punching through my stupid Comcast router! And I anticipate it will continue to work really well as I add more devices on my parent's, sibings', etc networks. It's cool to not have to think at all about where my devices are, since they're just all directly accessible through their dedicated IP on the wireguard VPN. I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!
mooncakes_ooohh 6y ago
I'm going to give this a try for personal use. For larger scale management alternative to ngrok I've been eyeing Shellhub for it's self-hosted capabilities and dashboard.
jeromenerf 6y ago
> I disabled nginx on the server and installed Caddy instead. I wonder how a somehow simpler config syntax (Caddyfile) compares to a simpler maintenance system (apt).
- rmoriz 6y ago
  Caddy underwent a complete rewrite that also invalidated most of the existing config. Personally I don't see any benefits in using caddy or another non mainstream httpd as it adds more risk to my project with usually very little benefits. Nothing against using it for hobby projects but in production I've migrated back to nginx.
  - megous 6y ago
    I had an itch to look at Caddy, after reading the comments, but breaking configuration backwards compatibility is a huge turn off.
    - mholt 6y ago
      v1 was an experiment. v2 is not.
    - miedpo 6y ago
      You should definitely take a look at it - Caddy works fantastically well. To give you an idea about the compatibility differences, it's like the upgrade from Vue.js 2 to Vue.js 3. Some changes to format are necessary for the new architecture. But you can always just run the older version (that's what I do). And swapping to the new one isn't terribly painful if your just using vanilla (for Caddy 1 extensions, which weren't created by the developer himself most of the time, I'm sure Caddy 2 versions will pop up over time).
    - anderspitman 6y ago
      Personally I prefer software that improves (slowly) over time, for at least a couple iterations. It's a balancing act to be sure, and there are diminishing returns to changing things up, but I think very little software comes out of the gate in a >80% optimal form. I haven't found the changes in Caddy's config format to be problematic in the least.
cft 6y ago
If you are on AT&T u-verse you can just assign a part of your IPv6 subnet (which appears to be static) to your RPi and disable the firewall to that host. Then you can accept incoming connections on ports 80, 22 and even 25 (for 25 I had to send a chat message in my AT&t account and it was opened the same day). Then you can have a full IPv6 web server at no additional cost to the price of your u-verse home internet.
- anderspitman 6y ago
  One downside is the actual steps are going to be different for every ISP/router combination. Tunneling is a universal solution, and provides other interesting tradeoffs.
  - cft 6y ago
    but the upside is that you truly have decentralized internet
    - anderspitman 6y ago
      Do you? Is having your ISP terminate your public IP better than having DigitalOcean or Hetzner terminate it?
deccanchargers 6y ago
Look at https://h2o.examp1e.net/ https://h2o.examp1e.net/ H2O web server seems to be faster than nginx , litespeed and supports http1/http2/http3 . It's config file is yaml file. For HTTPS, i recommend https://acme.sh https://acme.sh . It's very very easy and autorenews certificates using cron job.
- viraptor 6y ago
  When you're tunneling VPS to a home connection and rpi, the performance of the rev-proxy is not going to be a concern. Anything including a trivial script implementation of the proxy will be able to saturate this setup.
- anderspitman 6y ago
  Lots of mention of server push. Does anybody actually use it in production?
alexellisuk 6y ago
Ww have a much easier way to do this with inlets - https://docs.inlets.dev/#/get-started/quickstart-tcp-ssh?id=quick-start-tunnel-a-private-ssh-server-over-inlets-pro https://docs.inlets.dev/#/get-started/quickstart-tcp-ssh?id=...
- voltagex_ 6y ago
  >You will need an inlets PRO license, start a 14-day free trial.
  - alexellisuk 6y ago
    Why do you view paid software as somehow invalid or lesser than (a bunch of open source tools hacked together), or freemium SaaS products mentioned?
    - voltagex_ 6y ago
      I don't, necessarily, but it might have been nice to be up front in your comment that you were advertising for your company.
      - alexellisuk 6y ago
        Inlets is free and open source.
    - jlgaddis 6y ago
      The S.O.P. around here is to disclose any affiliations when recommending products -- especially commercial ones. You failed to do that twice just in this thread. For many of us, it's a trust thing. -- ETA: Speaking of trust, I just read this FAQ on inlets.dev: > Does inlets PRO "call home"? > We trust our users to purchase the correct license for their usage, and in return we make licensing simple with a fixed-term license. So, a few minutes after mentioning that "it's a trust thing", I came across this very simple, to the point, "yes or no" question ... ... and yet I notice that you|your company) (couldn't|didn't), for whatever reason, actually answer the question. That, of course, makes me even more distrustful than I was a few minutes ago!
voltagex_ 6y ago
Check with your ISP to see if you have IPv6 available, and use that. I have a cheap VPS that I can jump through if I'm on an IPv4 only host.
- sneak 6y ago
  The solution in TFA works behind NAT and routers/firewalls, and has the added benefit of encrypting all of the traffic between the two machines thanks to SSH. I’m not sure people who aren’t network administrators should be putting public IPv6 all over their LAN behind their router, at least not until v6 is more common and routers have better inbound security features for such things (like how NAT functions today for v4).
  - quaintdev 6y ago
    NAT like security for IPv6 is just dumb. We should always configure firewall for v4 and V6. NAT is not for security!! It has different purpose which is now being taken care by V6 and NAT should just die.
    - franga2000 6y ago
      Yes, BUT: for many many years, home networks have relied primarily on NAT for security. The firewall was only ever touched when doing something advanced and even people who were doing things like setting up home servers and such rarely had to deal with it. Every single tutorial on the topic is IPv4-specific and until IPv6 isn't more common, people will not write guides and tutorials with IPv6 in mind. So until the knowledge of proper security on open networks is common enough among home tinkerers, we should not be recommending people just throw everything straight onto the open Internet.
      - mercora 6y ago
        i don't think the typical NAT setup differs too much from a typical filtering setup. That is, only allow return traffic from public interfaces. While that is kinda implicit using NAT the change to being explicit is trivial. somewhat simplified like this using iptables on linux: iptables -t nat -A POSTROUTING -o WAN -j MASQUERADE vs iptables -A FORWARD -i WAN -m state ! --state RELATED,ESTABLISHED -j REJECT
belzebalex 6y ago
An esoteric alternative is to host a Tor hidden service. Tor hidden services don't require you open ports, and you don't need a static IP address. Your residential IP is protected by the onion router, and the .onion domain name is free!
- hnarn 6y ago
  > the .onion domain name is free! I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
  - loa_in_ 6y ago
    But .local aren't resolved by any public DNS server, so when outside your bubble you won't have them pointing where you expect while Tor names resolve whenever you can get connected to Tor Network
    - hnarn 6y ago
      Yeah, my point was just that if your host name is not a part of any “normal” or public DNS resolution, the point of it being free is kind of moot.
  - swiley 6y ago
    .local isn't part of a global namespace, it's part of a local namespace. Sure you don't get an interesting name but you do get one.
    - oarsinsync 6y ago
      .local isn’t the best comparison to .onion .oarsinsync is probably a better comparison. It’s not explicitly scoped anywhere, and it works nicely in my oarsinsync bubble, much like .onion works nicely in the ToR bubble. In both cases, you need to connect using some kind of tunnelling protocol to connect into another network.
    - 6y ago
nurettin 6y ago
I install a reverse proxy on every remote machine (raspberry pi or odroid) I manage. These machines are usually tucked behind someone's router, and it is hard to tell every router owner to forward the ssh port. When I install a reverse proxy, the machines can proactively tunnel right into a beacon machine whenever they are online and I can ssh into them from the beacon machine directly. I usually set up frpc as a service on the remote machine. It is also possible to use autossh with a monitoring port for this, but I noticed that frpc can bypass firewall restrictions much easier. This scheme also lets me use tools like Cockpit to view the health and status of my remotes. https://github.com/fatedier/frp https://github.com/fatedier/frp
- kzrdude 6y ago
  Just so that I understand the setup, this still requires one intermediate box - the beacon machine - to be running?
  - creztoe 6y ago
    That's how I understand it. Basically you have a machine hosted somewhere with a dedicated IP so you can access all remote machines from anywhere at any time, as long as they are connected to it via reverse proxy.
  - nurettin 6y ago
    It does require the beacon machine to be up, but the system easily recovers from a reboot of the beacon, since frpc will keep on trying to reconnect. It is a poor man's firewall-resilient one-way vpn.
- xrendan 6y ago
  Oooh, thanks for this, I'm currently using the autossh route but have had some issues with it. frp looks like it could solve my problems :)
- wil421 6y ago
  What are you using for your beacon machine? Do you have a static IP or a domain that your beacon uses?
mobilio 6y ago
This can be optimized easy with AutoSSH or "VPN TUN/TAP over SSH"
gprasanth 6y ago
Had a similar setup going. What next? Well I setup some handy shell functions: 1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily. 2) serve static files the same way, if needed too. the magic's all in the shell scripts. 3) explore new open source webapps running on local docker. Further? Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc. Limit's your imagination. But, security is also very important.
- anderspitman 6y ago
  Found the /r/selfhosted member.
dominicl 6y ago
One can also use the Diode P2P network to get the same without a server: https://support.diode.io/article/ss32engxlq https://support.diode.io/article/ss32engxlq works for ssh remote access, websites and stuff similar to ngrok just without the strings attached.
- anderspitman 6y ago
  There's got to be a public IP somewhere. What are the "exit nodes" for Diode?
johtso 6y ago
I just set up a raspberry pi with ZeroTier. So far it seems like a really easy way to make the pi accessible whatever network it is in, and took minimal configuration to have it forward requests to any of the other devices on the internal network. The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..
tiku 6y ago
Autossh for the tunnel works great.
fooblat 6y ago
As there isn't a single thing that is Raspberry Pi specific here, the title would be better written as Tunnel Public Internet Traffic To A Home Linux System via An Intermediary SSH Server
floatingsmoke 6y ago
I use Cloudflare Argo Tunnel for this purpose and am very satisfied. It's easy to setup and if you have TLD on Cloudflare, the daemon(cloudflared) creates new A record for each host and creates a tunnel between your host and the nearest Cloudflare data center. You can also run multiple services on a single host. And all these connections are secure.
- rexfuzzle 6y ago
  $5 per month and 10c per GB or am I missing something?
  - anderspitman 6y ago
    That's what I understand. They could corner this market if they made it flat $5-$10/mo and finished getting their DNS registrar up and running. But I suspect they have bigger fish to fry.
  - floatingsmoke 6y ago
    When you deal with tens of Raspberry Pis, require high availability, and have low traffic bandwidth Argo Tunnel makes really sense.
culopatin 6y ago
Side conversation: I'm interested in being pretty good at this sort of thing: setting up servers, making them connect, understand the protocols enough to put them to practice. I believe this is in the realm of IT. What are some good resources to read to get to my goal? Thanks!
- nsomaru 6y ago
  A thorough understanding of the OSI model would go a long way
- Craighead 6y ago
  NetworkChuck, Elithecomputerguy, Level1 Techs
- jlgaddis 6y ago
  Routing TCP/IP, Vol. 1 & 2.
- anderspitman 6y ago
  I'd recommend doing almost exactly what OP describes. Start on your local network first. Get a raspberry pi, and figure out how to set up a webserver on it you can access from other computers on your network.
bpizzi 6y ago
I don't get it, I may be missing some context thought: why the author didn't go for running said webapps directly on the VPS in the first place? Why bother running it on the Pi and set up the tunnel (which is a nice hack in itself, I get that)?
- lights0123 6y ago
  Something that requires LAN connectivity: media library off a local NAS (e.g. Plex), generic storage (e.g. Nextcloud), or home automation (e.g. Home Assistant).
  - berkes 6y ago
    Interesting solution. Also as alternative to a VPN. A VPN is often just too complex to set up, and also requires some hardware in your local network; though often your router will have that already. Off course a Rpi can act as VPN gateway/server. But solving it by flipping the "servers" around and running the "appserver" in your network then tunneling it to remote seems a really neat hack to avoid going into VPN hell.
  - flatiron 6y ago
    Just VPN the VPS? At least my router supports openvpn right on the device. Generates the config for you and everything.
- crusso 6y ago
  Perhaps the Raspberry Pi is used to control a robot, collect sensor data, whatever. Perhaps it was just for the fun of it. I've used similar techniques for doing a wide variety of operational tasks, so it can be worth it to take the time to figure out how to easily set up tunnels and reverse proxies.
- bityard 6y ago
  One use case is the cost of RAM when you buy a VPS. Low-end VPSes are cheap ($5 a month and under) but if you have multiple apps (or one big one), getting a decent amount of RAM in a VPS raises the price significantly. Using Linode as a benchmark, a 512 MB VPS is $5/mo but an 8 GB VPS is $40/mo. But an 8 GB Rasberry Pi 4 is well under $100 with peripherals so after three months it will have paid for itself. This of course assumes: 1. that the cost of powering the Pi is negligible 2. your home internet connection is stable enough and fast enough 3. your apps run as comfortably on the Pi as on VPS
upgradess 6y ago
Having used SSH for this purpose for some time, I decided to build an automatic service for easing reverse HTTP, HTTPS, and SSH ProxyJump through SSH tunnels, its available on Github as well, but if you'd like, try it out: ssh -R80:localhost:80 remote.moe it should output an URL where your local port 80 is now available https://github.com/fasmide/remotemoe https://github.com/fasmide/remotemoe
jakuboboza 6y ago
You could do same with free tier google instance micro. The thing is, what do you get from exposing your own pi to internet ? Like you are not going to host websites and shit so what crude home automation and monitoring system ?
- easton 6y ago
  Why not? Maybe not for a client, but for a blog or static site it takes just about no compute to host that, especially if it's behind Cloudflare or something.
voidmain0001 6y ago
SSH tunneling as a Service (STAAS): holepunch.io announced itself on HN in 2019 [1]. For now its free so it comes slightly ahead of using the Hetzner server. [1] https://news.ycombinator.com/item?id=20006785 https://news.ycombinator.com/item?id=20006785
jsiebens 6y ago
I'm a happy user of inlets OSS and inlets PRO for such use cases. Not only for the tunnelling, but also because the supporting tools, like inletsctl, which makes it really convenient to build such a tunnel with a few commands
fareesh 6y ago
My personal assistant runs on my pi. When I'm out I can use its chat app to talk to it. The chat uses a cloud real-time db like firebase. If I want to connect I just ask it to setup a reverse proxy. It can also turn on my machines via wake on lan. I'm quite happy with the setup since I can leave home anytime for long periods and always know I have access to the "home base".
alexellisuk 6y ago
There's a fairly easy way to do this with NGrok, and inlets makes it a little better by allowing you to have any port you like and a custom domain if needed. Checkout the tutorials at: https://docs.inlets.dev/ https://docs.inlets.dev/
driverdan 6y ago
I did something similar with Wireguard but wasn't able to get it to do what I need. I'm trying to give my VPS access to other computers on my home network. The Pi and my VPS can communicate with each other just fine. For some reason the pi won't forward any packets from the VPS, they just seem to drop. I've scoured all config that I could think of, iptables, network adaptors, Wireguard, but can't find the cause. Anyone encounter this or have a place to point me for help?
- johtso 6y ago
  I set something like this up the other day using ZeroTier rather than Wireguard. Followed the instructions here and it "just worked" https://zerotier.atlassian.net/wiki/spaces/SD/pages/224395274/Route+between+ZeroTier+and+Physical+Networks https://zerotier.atlassian.net/wiki/spaces/SD/pages/22439527... The key was enabling IP forwarding, and setting the right iptables rules.
- granra 6y ago
  Is ip_forward[1] enabled on the rpi? Also is your home network's cidr in allowed ips on the vps' wireguard interface? [1]: https://docs.mysirena.xyz/centos-7/prefilight-configuration/how-to-enable-ip-forwarding https://docs.mysirena.xyz/centos-7/prefilight-configuration/...
  - driverdan 6y ago
    Yes, IP forwarding is enabled and my home network is set under AllowedIPs.
    - granra 6y ago
      You can trace log iptables and see first if the wireguard traffic is being blocked by some rule `iptables -t raw -I PREROUTING -p udp --dport 51820 -j TRACE` (assuming default 51820 port) and look at dmesg or `journalctl -f` If that is not being blocked then you can look at `iptables -t raw -I PREROUTING -s <vps' wireguard ip> -j TRACE` and try pinging or reaching a service on your home network and look at dmesg again. Just don't forget to remove these rules again as they will continue to log. Beyond that, I'm not sure.
antoniomika 6y ago
Up until about 2 years ago, I used to use the same method as you've listed! Actually, I still use this method, but do it a bit differently. I have a ZeroTier network setup to connect a VM and RPI for internal communication, then I use nginx with the stream module [1] in order to do SNI proxying down to the RPI from the VM over the internal ZT network. Why use the stream module vs the normal proxy module you might ask? You can forward TLS connections down to an edge device without needing to have the TLS certs on the cloud server. This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine. This also has the added benefit of allowing me to forward full wildcard subdomains down to the edge device without having to create a new config entry for it each time. Now, I use (shameless self plug) sish [2] to set this up. It uses SSH, so I get the added benefit of all of the above, but I don't really worry about using an internal network for ingress communication. It's also very easy to share the service with friends/coworkers so they can get the added benefit of it too. I also make use of the loadbalancing feature for test pi clusters when doing distributed testing which is a breeze now. There are a bunch of these popping up now which I'm happy to see (serveo, remotemoe, even ngrok has a SSH gateway!) so there's a ton of competition here. I definitely recommend checking at least one of them out since they're pretty easy to setup and can be used in a variety of ways! [1] http://nginx.org/en/docs/stream/ngx_stream_core_module.html http://nginx.org/en/docs/stream/ngx_stream_core_module.html [2] https://github.com/antoniomika/sish https://github.com/antoniomika/sish
- forty 6y ago
  > This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine If you share a server with your domain name pointing on it, can't just other people on the server use HTTP validation to get a valid certificate for your domain, then do mitm?
  - antoniomika 6y ago
    Reading into it more than I was going for :) I firewall all traffic over port 80, so HTTP validation won't work. I also don't actually share the server with anyone that has access to it, it's moreso so I can provide them a similar setup to what I have with some semblance of security even if they don't have the expertise to do so.
neurostimulant 6y ago
I have a similar setup as a fallback method to make sure I can always access my local server at home on any network (the primary methods are using zerotier and opening a port on my router), but I used autossh managed by supervisord to make sure the ssh tunnel stay up. Without autossh, the ssh tunnel sometimes randomly closed. I tried configuring ServerAliveInterval to some low value but the issue still happen sometimes, and when it happened sometimes the port it binds on the vps can't be reused anymore (port in use error). Autossh fixed all of those issues.
syoc 6y ago
At this point you are pretty close to just being able to use your VPS for whatever you wanted to do access remotely. I have tried a ssh port forward solution earlier, but the connection fell down, did not always come up after boot and was generally unreliable. I found wireguard to be much easier to set up and highly reliable. It's in my own opinion not that hard to set up either, there are plenty of guides. The wireguard project docs are also quite good.
- anderspitman 6y ago
  > I have tried a ssh port forward solution earlier, but the connection fell down, did not always come up after boot and was generally unreliable. WireGuard is a great solution, but I have heard people speak highly of autossh, which I believe may solve this.
eximius 6y ago
I wish we were a little farther along so I'd have more to say, but we're hoping to solve this with https://hoppy.network/ https://hoppy.network/. It's nothing you couldn't do yourself, but we host it for you so you don't have to. We host WireGuard servers which assign a stable public IPv4 and IPv6 on your device's WireGuard tunnel interface. As far as your device is concerned, it's like its tunnel interface is a publicly addressable connection. Because it is! Just with one extra encrypted hop at the end. Nothing to install besides WireGuard itself. WireGuard's biggest papercut (Dynamic DNS) is solved because we provide you a connection to a server with a static IP. No promotional emails. No spam. Just a simple service for a specific, niche need. Hope to post about its launch here soon. (logan@hoppy.network for questions!)
- anderspitman 6y ago
  I like it. Forwarding public ips through WireGuard is a nice level of abstraction. I maintain a list of tunneling solutions here: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling Happy to add hoppy if you think you're a good fit and submit a PR. A couple questions: * Do you provide any sort of reverse proxy service, ie HTTPS termination with Let's Encrypt support, similar to OP? * Do you provide any DNS integration or automation, or purely IPs? EDIT: Also how would you compare hoppy to Tailscale?
  - rubatuga 6y ago
    Yes, we provide reverse DNS for the IPv4 through the management interface. A subdomain (*.hoppy.link) for the tunnelled IP address provided can be created if you send us a support email. We do not provide any reverse proxy, but it could be a possibility if there is demand. Tailscale uses the non-routable IP space, and is strictly for private networks, and cannot publish to public networks.
- athriren 6y ago
  about how far out do you think you are?
  - 6y ago
anderspitman 6y ago
I maintain a list of solutions to this problem: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling. There are a surprising number of tools that all do essentially the same thing, and it's often hard to tell how they're different from the other 30 options. Making a really good solution to this problem has been the focus of my free time for the past month since I wrote this comment[0]. The project I'm working on: * 100% open source and MIT licensed * Strive for simplicity throughout. Very few dependencies. No config files, and only a single CLI parameter required to run the server. * Clients can be remote controlled by the server, and each client can open an arbitrary number of tunnels. ie you don't have to start a separate ngrok process for each tunnel. Just set up the client on each machine you want to tunnel out of and forget about it. Everything is controlled through a simple web UI (or REST api). * Automate as much as possible. Tunnel creation and Let's Encrypt are already working. Eventually you'll be able to add an API token from your DNS provider and that'll be automated as well. It's not quite ready (launch planned for 31 Oct 2020), but you can check it out here: https://boringproxy.io/ https://boringproxy.io/ [0]: https://news.ycombinator.com/item?id=24475946 https://news.ycombinator.com/item?id=24475946
- mobilio 6y ago
  THANK YOU! I still miss Serveo because this was easiest tunnel solution that "just works!"
  - anderspitman 6y ago
    If you want a self-hosted alternative to serveo, I recommend sish (comprehensive) or my own SirTunnel (simple and minimal). If you just need ad-hoc tunnels on someone else's servers for free, I'm not sure what the leading options are. localhost.run is one. [0]: https://github.com/antoniomika/sish https://github.com/antoniomika/sish [1]: https://github.com/anderspitman/SirTunnel https://github.com/anderspitman/SirTunnel
  - tommyvan 6y ago
    I've seen a few servo refugees recently land on my service, https://localhost.run/ https://localhost.run/ I didn't get to try serveo before it ended but I _think_ I do something similar, "ssh -R 80:localhost:8080 ssh.localhost.run" will give you an internet accessible address tunneled to localhost:8080.
anderspitman 6y ago
> I set GatewayPorts clientspecified in my /etc/ssh/sshd_config on the server. This is needed so that the client (RaspberryPi) can specify which ports to enabled reverse tunneling on. This isn't quite what GatewayPorts does. By default, the client can open a tunnel to any ports the user has access to, but those ports can only be connected to from localhost on the server. So even if you don't have a firewall, external IPs can't connect to your tunnel (but your reverse proxy can, if it's running on the same machine). You can use GatewayPorts to allow binding from other IPs. This is useful if you want to tunnel arbitrary TCP traffic, not just HTTP through your reverse proxy.
aborsy 6y ago
A comparison of the security of these several approaches would be helpful. Am I correct that Wireguard and SSH are the more secure options?
- anderspitman 6y ago
  I would personally trust SSH based on OpenSSH the most, WireGuard second, custom SSH third, TLS 4th, and then the rest.
  - aborsy 6y ago
    Interesting! I thought Wireguard is 4K lines of code while OpenSSH is hundreds of thousands of lines of code. Thus Wireguard is more secure.
    - anderspitman 6y ago
      In 5-10 years WireGuard will likely be my #1, once it's had a chance to prove itself. I have no doubt it will, but you can still hide serious security issues in 4k LOC. Especially C. Also, you can still have bugs in client implementations, and especially in the process used to pass keys around. I picked OpenSSH purely because it's been around so long and had so much data tunneled through it. That said, I would still trust WG with my data. I just don't trust it more than a proven SSH implementation, yet.
tommyvan 6y ago
A few years ago I wrote a custom SSH server to do exactly this, but for webhook development rather than tunneling to a pi. other peeps found it useful so it's turned into a bit of a side project. If you want to have a go before spending time on setting up your own server run "ssh -R 80:localhost:8080 ssh.localhost.run" to tunnel to localhost:8080, you'll see an internet accessible domain returned to your terminal.

146 points by mathnmusic 6y ago

I think a better option would be to create a Wireguard tunnel between Raspberry Pi and the remote server instead of a SSH Tunnel. Then there is no need to add or change ports and restart the tunnel for every service.

thespoonbends 6y ago
OP's solution sound good enough for their needs. Solutions can always be improved, but it's not always worth doing that.
- iamd3vil 6y ago
  OP has literally written in the gist about exploring a way to map entire port range and avoiding doing this, so the non hacky way of doing this is setting up something like a wireguard tunnel. That's the reason I suggested doing this instead of a tunnel which has other disadvantages like doing TCP on TCP.
  - sneak 6y ago
    SSH tunnels do not run TCP inside of them, just the bytes of the connection data itself. The only TCP in use is the TCP connection of the SSH connection between hosts.
    - iamd3vil 6y ago
      Ohh TIL but my other point still stands.
    - xorcist 6y ago
      It used to be common (at least not unhead of) to run ppp over ssh, which has this problem.
mercora 6y ago
while i think this is true (any other vpn software would work too though) i want to point out that you actually can bring up a tun interface using ssh with the "-w local_tun[:remote_tun]" flag somewhat easily if you want to. It is also possible to make forwarding work in either direction using an integrated socks proxy using "-R" or "-D" flags ...
- anderspitman 6y ago
  > you actually can bring up a tun interface using ssh Beware of TCP over TCP issues[0] when using SSH for tun. [0]: http://sites.inka.de/bigred/devel/tcp-tcp.html http://sites.inka.de/bigred/devel/tcp-tcp.html

sigmaprimus 6y ago

Is your plan to run some sort of IOT application on the PI? I'm a little confused as to why you would go through the trouble of running a reverse proxy on a remote server that is fully capable of hosting your web applications and or just setting up a port forwarding rule on your local wifi router to redirect 10080 to 80.

thespoonbends 6y ago
I do a similar thing so I don't have to store sensitive data on a VPS. The sensitive data is used to compute non-sensitive results that I'm happy to go via a VPS.
3np 6y ago
Privacy; keep data local, without exposing your residential IP.
- BHSPitMonkey 6y ago
  Is exposing one's residential IP address such a bad thing? Is the threat model just that someone might wish to target you individually, and that they'll have better luck finding a vuln into your LAN if your services are listening directly on public ports?
  - 3np 6y ago
    There are unlimited imaginable reasons why one may want to de-correlate the data sent in or out from ones software to ones personal identity, yes.
    - BHSPitMonkey 6y ago
      I'd think that using a remote VPN is a better/broader solution to that particular problem. People don't often self-host things at the home, so it's unlikely anyone out there would see my DNS record and assume that it's of value and a useful tool for correlating my IP with my identity (and this actor would need to have eyes on a lot of different services' server logs in order to have data to _maybe_ correlate with me in the first place).

lhoff 6y ago

I have a similar setup but instead of SSH Tunnels i use wireguard. In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.

allthingsequal 6y ago
Could you please elaborate on your setup? I haven't had the opportunity to play with Wireguard yet, that could be a useful first project to me :-)
- hakeberio 6y ago
  Pivpn is beginner-friendly and you can always branch off from there. https://www.pivpn.io https://www.pivpn.io
- vsajip 6y ago
  I've started playing around with Tailscale [1], which is based on WireGuard. Free to use for a single user, and I've tried it out on Raspberry Pi, Linux x86_64, Windows and Android. Seems to work well, except there appears to be some issue with accessing Windows via the secure IP (neither ping nor Remote Desktop appear to get through the Windows firewall, despite having a specific firewall rule for Tailscale) [2]. Can SSH into e.g. Linux hosts using PuTTY from Windows without issues. [1] https://tailscale.com/ https://tailscale.com/ [2] https://github.com/tailscale/tailscale/issues/854 https://github.com/tailscale/tailscale/issues/854
- 8K832d7tNmiQ 6y ago
  the idea is that your hetzner server acts as a gateway and you can communicate with your pi through the virtual private network, therefore removing any need to access through public ip.
twicetwice 6y ago
I did the same. It works really well for hole-punching through my stupid Comcast router! And I anticipate it will continue to work really well as I add more devices on my parent's, sibings', etc networks. It's cool to not have to think at all about where my devices are, since they're just all directly accessible through their dedicated IP on the wireguard VPN. I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!

mooncakes_ooohh 6y ago

I'm going to give this a try for personal use. For larger scale management alternative to ngrok I've been eyeing Shellhub for it's self-hosted capabilities and dashboard.

jeromenerf 6y ago

> I disabled nginx on the server and installed Caddy instead. I wonder how a somehow simpler config syntax (Caddyfile) compares to a simpler maintenance system (apt).

rmoriz 6y ago
Caddy underwent a complete rewrite that also invalidated most of the existing config. Personally I don't see any benefits in using caddy or another non mainstream httpd as it adds more risk to my project with usually very little benefits. Nothing against using it for hobby projects but in production I've migrated back to nginx.
- megous 6y ago
  I had an itch to look at Caddy, after reading the comments, but breaking configuration backwards compatibility is a huge turn off.
  - mholt 6y ago
    v1 was an experiment. v2 is not.
  - miedpo 6y ago
    You should definitely take a look at it - Caddy works fantastically well. To give you an idea about the compatibility differences, it's like the upgrade from Vue.js 2 to Vue.js 3. Some changes to format are necessary for the new architecture. But you can always just run the older version (that's what I do). And swapping to the new one isn't terribly painful if your just using vanilla (for Caddy 1 extensions, which weren't created by the developer himself most of the time, I'm sure Caddy 2 versions will pop up over time).
  - anderspitman 6y ago
    Personally I prefer software that improves (slowly) over time, for at least a couple iterations. It's a balancing act to be sure, and there are diminishing returns to changing things up, but I think very little software comes out of the gate in a >80% optimal form. I haven't found the changes in Caddy's config format to be problematic in the least.

cft 6y ago

If you are on AT&T u-verse you can just assign a part of your IPv6 subnet (which appears to be static) to your RPi and disable the firewall to that host. Then you can accept incoming connections on ports 80, 22 and even 25 (for 25 I had to send a chat message in my AT&t account and it was opened the same day). Then you can have a full IPv6 web server at no additional cost to the price of your u-verse home internet.

anderspitman 6y ago
One downside is the actual steps are going to be different for every ISP/router combination. Tunneling is a universal solution, and provides other interesting tradeoffs.
- cft 6y ago
  but the upside is that you truly have decentralized internet
  - anderspitman 6y ago
    Do you? Is having your ISP terminate your public IP better than having DigitalOcean or Hetzner terminate it?

deccanchargers 6y ago

Look at https://h2o.examp1e.net/ https://h2o.examp1e.net/ H2O web server seems to be faster than nginx , litespeed and supports http1/http2/http3 . It's config file is yaml file. For HTTPS, i recommend https://acme.sh https://acme.sh . It's very very easy and autorenews certificates using cron job.

viraptor 6y ago
When you're tunneling VPS to a home connection and rpi, the performance of the rev-proxy is not going to be a concern. Anything including a trivial script implementation of the proxy will be able to saturate this setup.
anderspitman 6y ago
Lots of mention of server push. Does anybody actually use it in production?

alexellisuk 6y ago

Ww have a much easier way to do this with inlets - https://docs.inlets.dev/#/get-started/quickstart-tcp-ssh?id=quick-start-tunnel-a-private-ssh-server-over-inlets-pro https://docs.inlets.dev/#/get-started/quickstart-tcp-ssh?id=...

voltagex_ 6y ago
>You will need an inlets PRO license, start a 14-day free trial.
- alexellisuk 6y ago
  Why do you view paid software as somehow invalid or lesser than (a bunch of open source tools hacked together), or freemium SaaS products mentioned?
  - voltagex_ 6y ago
    I don't, necessarily, but it might have been nice to be up front in your comment that you were advertising for your company.
    - alexellisuk 6y ago
      Inlets is free and open source.
  - jlgaddis 6y ago
    The S.O.P. around here is to disclose any affiliations when recommending products -- especially commercial ones. You failed to do that twice just in this thread. For many of us, it's a trust thing. -- ETA: Speaking of trust, I just read this FAQ on inlets.dev: > Does inlets PRO "call home"? > We trust our users to purchase the correct license for their usage, and in return we make licensing simple with a fixed-term license. So, a few minutes after mentioning that "it's a trust thing", I came across this very simple, to the point, "yes or no" question ... ... and yet I notice that you|your company) (couldn't|didn't), for whatever reason, actually answer the question. That, of course, makes me even more distrustful than I was a few minutes ago!

voltagex_ 6y ago

Check with your ISP to see if you have IPv6 available, and use that. I have a cheap VPS that I can jump through if I'm on an IPv4 only host.

sneak 6y ago
The solution in TFA works behind NAT and routers/firewalls, and has the added benefit of encrypting all of the traffic between the two machines thanks to SSH. I’m not sure people who aren’t network administrators should be putting public IPv6 all over their LAN behind their router, at least not until v6 is more common and routers have better inbound security features for such things (like how NAT functions today for v4).
- quaintdev 6y ago
  NAT like security for IPv6 is just dumb. We should always configure firewall for v4 and V6. NAT is not for security!! It has different purpose which is now being taken care by V6 and NAT should just die.
  - franga2000 6y ago
    Yes, BUT: for many many years, home networks have relied primarily on NAT for security. The firewall was only ever touched when doing something advanced and even people who were doing things like setting up home servers and such rarely had to deal with it. Every single tutorial on the topic is IPv4-specific and until IPv6 isn't more common, people will not write guides and tutorials with IPv6 in mind. So until the knowledge of proper security on open networks is common enough among home tinkerers, we should not be recommending people just throw everything straight onto the open Internet.
    - mercora 6y ago
      i don't think the typical NAT setup differs too much from a typical filtering setup. That is, only allow return traffic from public interfaces. While that is kinda implicit using NAT the change to being explicit is trivial. somewhat simplified like this using iptables on linux: iptables -t nat -A POSTROUTING -o WAN -j MASQUERADE vs iptables -A FORWARD -i WAN -m state ! --state RELATED,ESTABLISHED -j REJECT

belzebalex 6y ago

An esoteric alternative is to host a Tor hidden service. Tor hidden services don't require you open ports, and you don't need a static IP address. Your residential IP is protected by the onion router, and the .onion domain name is free!

hnarn 6y ago
> the .onion domain name is free! I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
- loa_in_ 6y ago
  But .local aren't resolved by any public DNS server, so when outside your bubble you won't have them pointing where you expect while Tor names resolve whenever you can get connected to Tor Network
  - hnarn 6y ago
    Yeah, my point was just that if your host name is not a part of any “normal” or public DNS resolution, the point of it being free is kind of moot.
- swiley 6y ago
  .local isn't part of a global namespace, it's part of a local namespace. Sure you don't get an interesting name but you do get one.
  - oarsinsync 6y ago
    .local isn’t the best comparison to .onion .oarsinsync is probably a better comparison. It’s not explicitly scoped anywhere, and it works nicely in my oarsinsync bubble, much like .onion works nicely in the ToR bubble. In both cases, you need to connect using some kind of tunnelling protocol to connect into another network.
  - 6y ago

nurettin 6y ago

I install a reverse proxy on every remote machine (raspberry pi or odroid) I manage. These machines are usually tucked behind someone's router, and it is hard to tell every router owner to forward the ssh port. When I install a reverse proxy, the machines can proactively tunnel right into a beacon machine whenever they are online and I can ssh into them from the beacon machine directly. I usually set up frpc as a service on the remote machine. It is also possible to use autossh with a monitoring port for this, but I noticed that frpc can bypass firewall restrictions much easier. This scheme also lets me use tools like Cockpit to view the health and status of my remotes. https://github.com/fatedier/frp https://github.com/fatedier/frp

kzrdude 6y ago
Just so that I understand the setup, this still requires one intermediate box - the beacon machine - to be running?
- creztoe 6y ago
  That's how I understand it. Basically you have a machine hosted somewhere with a dedicated IP so you can access all remote machines from anywhere at any time, as long as they are connected to it via reverse proxy.
- nurettin 6y ago
  It does require the beacon machine to be up, but the system easily recovers from a reboot of the beacon, since frpc will keep on trying to reconnect. It is a poor man's firewall-resilient one-way vpn.
xrendan 6y ago
Oooh, thanks for this, I'm currently using the autossh route but have had some issues with it. frp looks like it could solve my problems :)
wil421 6y ago
What are you using for your beacon machine? Do you have a static IP or a domain that your beacon uses?

mobilio 6y ago

This can be optimized easy with AutoSSH or "VPN TUN/TAP over SSH"

gprasanth 6y ago

Had a similar setup going. What next? Well I setup some handy shell functions: 1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily. 2) serve static files the same way, if needed too. the magic's all in the shell scripts. 3) explore new open source webapps running on local docker. Further? Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc. Limit's your imagination. But, security is also very important.

anderspitman 6y ago
Found the /r/selfhosted member.

dominicl 6y ago

One can also use the Diode P2P network to get the same without a server: https://support.diode.io/article/ss32engxlq https://support.diode.io/article/ss32engxlq works for ssh remote access, websites and stuff similar to ngrok just without the strings attached.

anderspitman 6y ago
There's got to be a public IP somewhere. What are the "exit nodes" for Diode?

johtso 6y ago

I just set up a raspberry pi with ZeroTier. So far it seems like a really easy way to make the pi accessible whatever network it is in, and took minimal configuration to have it forward requests to any of the other devices on the internal network. The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..

tiku 6y ago

Autossh for the tunnel works great.

fooblat 6y ago

As there isn't a single thing that is Raspberry Pi specific here, the title would be better written as Tunnel Public Internet Traffic To A Home Linux System via An Intermediary SSH Server

floatingsmoke 6y ago

I use Cloudflare Argo Tunnel for this purpose and am very satisfied. It's easy to setup and if you have TLD on Cloudflare, the daemon(cloudflared) creates new A record for each host and creates a tunnel between your host and the nearest Cloudflare data center. You can also run multiple services on a single host. And all these connections are secure.

rexfuzzle 6y ago
$5 per month and 10c per GB or am I missing something?
- anderspitman 6y ago
  That's what I understand. They could corner this market if they made it flat $5-$10/mo and finished getting their DNS registrar up and running. But I suspect they have bigger fish to fry.
- floatingsmoke 6y ago
  When you deal with tens of Raspberry Pis, require high availability, and have low traffic bandwidth Argo Tunnel makes really sense.

culopatin 6y ago

Side conversation: I'm interested in being pretty good at this sort of thing: setting up servers, making them connect, understand the protocols enough to put them to practice. I believe this is in the realm of IT. What are some good resources to read to get to my goal? Thanks!

nsomaru 6y ago
A thorough understanding of the OSI model would go a long way
Craighead 6y ago
NetworkChuck, Elithecomputerguy, Level1 Techs
jlgaddis 6y ago
Routing TCP/IP, Vol. 1 & 2.
anderspitman 6y ago
I'd recommend doing almost exactly what OP describes. Start on your local network first. Get a raspberry pi, and figure out how to set up a webserver on it you can access from other computers on your network.

bpizzi 6y ago

I don't get it, I may be missing some context thought: why the author didn't go for running said webapps directly on the VPS in the first place? Why bother running it on the Pi and set up the tunnel (which is a nice hack in itself, I get that)?

lights0123 6y ago
Something that requires LAN connectivity: media library off a local NAS (e.g. Plex), generic storage (e.g. Nextcloud), or home automation (e.g. Home Assistant).
- berkes 6y ago
  Interesting solution. Also as alternative to a VPN. A VPN is often just too complex to set up, and also requires some hardware in your local network; though often your router will have that already. Off course a Rpi can act as VPN gateway/server. But solving it by flipping the "servers" around and running the "appserver" in your network then tunneling it to remote seems a really neat hack to avoid going into VPN hell.
- flatiron 6y ago
  Just VPN the VPS? At least my router supports openvpn right on the device. Generates the config for you and everything.
crusso 6y ago
Perhaps the Raspberry Pi is used to control a robot, collect sensor data, whatever. Perhaps it was just for the fun of it. I've used similar techniques for doing a wide variety of operational tasks, so it can be worth it to take the time to figure out how to easily set up tunnels and reverse proxies.
bityard 6y ago
One use case is the cost of RAM when you buy a VPS. Low-end VPSes are cheap ($5 a month and under) but if you have multiple apps (or one big one), getting a decent amount of RAM in a VPS raises the price significantly. Using Linode as a benchmark, a 512 MB VPS is $5/mo but an 8 GB VPS is $40/mo. But an 8 GB Rasberry Pi 4 is well under $100 with peripherals so after three months it will have paid for itself. This of course assumes: 1. that the cost of powering the Pi is negligible 2. your home internet connection is stable enough and fast enough 3. your apps run as comfortably on the Pi as on VPS

upgradess 6y ago

Having used SSH for this purpose for some time, I decided to build an automatic service for easing reverse HTTP, HTTPS, and SSH ProxyJump through SSH tunnels, its available on Github as well, but if you'd like, try it out: ssh -R80:localhost:80 remote.moe it should output an URL where your local port 80 is now available https://github.com/fasmide/remotemoe https://github.com/fasmide/remotemoe

jakuboboza 6y ago

You could do same with free tier google instance micro. The thing is, what do you get from exposing your own pi to internet ? Like you are not going to host websites and shit so what crude home automation and monitoring system ?

easton 6y ago
Why not? Maybe not for a client, but for a blog or static site it takes just about no compute to host that, especially if it's behind Cloudflare or something.

voidmain0001 6y ago

SSH tunneling as a Service (STAAS): holepunch.io announced itself on HN in 2019 [1]. For now its free so it comes slightly ahead of using the Hetzner server. [1] https://news.ycombinator.com/item?id=20006785 https://news.ycombinator.com/item?id=20006785

jsiebens 6y ago

I'm a happy user of inlets OSS and inlets PRO for such use cases. Not only for the tunnelling, but also because the supporting tools, like inletsctl, which makes it really convenient to build such a tunnel with a few commands

fareesh 6y ago

My personal assistant runs on my pi. When I'm out I can use its chat app to talk to it. The chat uses a cloud real-time db like firebase. If I want to connect I just ask it to setup a reverse proxy. It can also turn on my machines via wake on lan. I'm quite happy with the setup since I can leave home anytime for long periods and always know I have access to the "home base".

alexellisuk 6y ago

There's a fairly easy way to do this with NGrok, and inlets makes it a little better by allowing you to have any port you like and a custom domain if needed. Checkout the tutorials at: https://docs.inlets.dev/ https://docs.inlets.dev/

driverdan 6y ago

I did something similar with Wireguard but wasn't able to get it to do what I need. I'm trying to give my VPS access to other computers on my home network. The Pi and my VPS can communicate with each other just fine. For some reason the pi won't forward any packets from the VPS, they just seem to drop. I've scoured all config that I could think of, iptables, network adaptors, Wireguard, but can't find the cause. Anyone encounter this or have a place to point me for help?

johtso 6y ago
I set something like this up the other day using ZeroTier rather than Wireguard. Followed the instructions here and it "just worked" https://zerotier.atlassian.net/wiki/spaces/SD/pages/224395274/Route+between+ZeroTier+and+Physical+Networks https://zerotier.atlassian.net/wiki/spaces/SD/pages/22439527... The key was enabling IP forwarding, and setting the right iptables rules.
granra 6y ago
Is ip_forward[1] enabled on the rpi? Also is your home network's cidr in allowed ips on the vps' wireguard interface? [1]: https://docs.mysirena.xyz/centos-7/prefilight-configuration/how-to-enable-ip-forwarding https://docs.mysirena.xyz/centos-7/prefilight-configuration/...
- driverdan 6y ago
  Yes, IP forwarding is enabled and my home network is set under AllowedIPs.
  - granra 6y ago
    You can trace log iptables and see first if the wireguard traffic is being blocked by some rule `iptables -t raw -I PREROUTING -p udp --dport 51820 -j TRACE` (assuming default 51820 port) and look at dmesg or `journalctl -f` If that is not being blocked then you can look at `iptables -t raw -I PREROUTING -s <vps' wireguard ip> -j TRACE` and try pinging or reaching a service on your home network and look at dmesg again. Just don't forget to remove these rules again as they will continue to log. Beyond that, I'm not sure.

antoniomika 6y ago

Up until about 2 years ago, I used to use the same method as you've listed! Actually, I still use this method, but do it a bit differently. I have a ZeroTier network setup to connect a VM and RPI for internal communication, then I use nginx with the stream module [1] in order to do SNI proxying down to the RPI from the VM over the internal ZT network. Why use the stream module vs the normal proxy module you might ask? You can forward TLS connections down to an edge device without needing to have the TLS certs on the cloud server. This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine. This also has the added benefit of allowing me to forward full wildcard subdomains down to the edge device without having to create a new config entry for it each time. Now, I use (shameless self plug) sish [2] to set this up. It uses SSH, so I get the added benefit of all of the above, but I don't really worry about using an internal network for ingress communication. It's also very easy to share the service with friends/coworkers so they can get the added benefit of it too. I also make use of the loadbalancing feature for test pi clusters when doing distributed testing which is a breeze now. There are a bunch of these popping up now which I'm happy to see (serveo, remotemoe, even ngrok has a SSH gateway!) so there's a ton of competition here. I definitely recommend checking at least one of them out since they're pretty easy to setup and can be used in a variety of ways! [1] http://nginx.org/en/docs/stream/ngx_stream_core_module.html http://nginx.org/en/docs/stream/ngx_stream_core_module.html [2] https://github.com/antoniomika/sish https://github.com/antoniomika/sish

forty 6y ago
> This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine If you share a server with your domain name pointing on it, can't just other people on the server use HTTP validation to get a valid certificate for your domain, then do mitm?
- antoniomika 6y ago
  Reading into it more than I was going for :) I firewall all traffic over port 80, so HTTP validation won't work. I also don't actually share the server with anyone that has access to it, it's moreso so I can provide them a similar setup to what I have with some semblance of security even if they don't have the expertise to do so.

neurostimulant 6y ago

I have a similar setup as a fallback method to make sure I can always access my local server at home on any network (the primary methods are using zerotier and opening a port on my router), but I used autossh managed by supervisord to make sure the ssh tunnel stay up. Without autossh, the ssh tunnel sometimes randomly closed. I tried configuring ServerAliveInterval to some low value but the issue still happen sometimes, and when it happened sometimes the port it binds on the vps can't be reused anymore (port in use error). Autossh fixed all of those issues.

syoc 6y ago

At this point you are pretty close to just being able to use your VPS for whatever you wanted to do access remotely. I have tried a ssh port forward solution earlier, but the connection fell down, did not always come up after boot and was generally unreliable. I found wireguard to be much easier to set up and highly reliable. It's in my own opinion not that hard to set up either, there are plenty of guides. The wireguard project docs are also quite good.

anderspitman 6y ago
> I have tried a ssh port forward solution earlier, but the connection fell down, did not always come up after boot and was generally unreliable. WireGuard is a great solution, but I have heard people speak highly of autossh, which I believe may solve this.

eximius 6y ago

I wish we were a little farther along so I'd have more to say, but we're hoping to solve this with https://hoppy.network/ https://hoppy.network/. It's nothing you couldn't do yourself, but we host it for you so you don't have to. We host WireGuard servers which assign a stable public IPv4 and IPv6 on your device's WireGuard tunnel interface. As far as your device is concerned, it's like its tunnel interface is a publicly addressable connection. Because it is! Just with one extra encrypted hop at the end. Nothing to install besides WireGuard itself. WireGuard's biggest papercut (Dynamic DNS) is solved because we provide you a connection to a server with a static IP. No promotional emails. No spam. Just a simple service for a specific, niche need. Hope to post about its launch here soon. (logan@hoppy.network for questions!)

anderspitman 6y ago
I like it. Forwarding public ips through WireGuard is a nice level of abstraction. I maintain a list of tunneling solutions here: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling Happy to add hoppy if you think you're a good fit and submit a PR. A couple questions: * Do you provide any sort of reverse proxy service, ie HTTPS termination with Let's Encrypt support, similar to OP? * Do you provide any DNS integration or automation, or purely IPs? EDIT: Also how would you compare hoppy to Tailscale?
- rubatuga 6y ago
  Yes, we provide reverse DNS for the IPv4 through the management interface. A subdomain (*.hoppy.link) for the tunnelled IP address provided can be created if you send us a support email. We do not provide any reverse proxy, but it could be a possibility if there is demand. Tailscale uses the non-routable IP space, and is strictly for private networks, and cannot publish to public networks.
athriren 6y ago
about how far out do you think you are?
- 6y ago

anderspitman 6y ago

I maintain a list of solutions to this problem: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling. There are a surprising number of tools that all do essentially the same thing, and it's often hard to tell how they're different from the other 30 options. Making a really good solution to this problem has been the focus of my free time for the past month since I wrote this comment[0]. The project I'm working on: * 100% open source and MIT licensed * Strive for simplicity throughout. Very few dependencies. No config files, and only a single CLI parameter required to run the server. * Clients can be remote controlled by the server, and each client can open an arbitrary number of tunnels. ie you don't have to start a separate ngrok process for each tunnel. Just set up the client on each machine you want to tunnel out of and forget about it. Everything is controlled through a simple web UI (or REST api). * Automate as much as possible. Tunnel creation and Let's Encrypt are already working. Eventually you'll be able to add an API token from your DNS provider and that'll be automated as well. It's not quite ready (launch planned for 31 Oct 2020), but you can check it out here: https://boringproxy.io/ https://boringproxy.io/ [0]: https://news.ycombinator.com/item?id=24475946 https://news.ycombinator.com/item?id=24475946

mobilio 6y ago
THANK YOU! I still miss Serveo because this was easiest tunnel solution that "just works!"
- anderspitman 6y ago
  If you want a self-hosted alternative to serveo, I recommend sish (comprehensive) or my own SirTunnel (simple and minimal). If you just need ad-hoc tunnels on someone else's servers for free, I'm not sure what the leading options are. localhost.run is one. [0]: https://github.com/antoniomika/sish https://github.com/antoniomika/sish [1]: https://github.com/anderspitman/SirTunnel https://github.com/anderspitman/SirTunnel
- tommyvan 6y ago
  I've seen a few servo refugees recently land on my service, https://localhost.run/ https://localhost.run/ I didn't get to try serveo before it ended but I _think_ I do something similar, "ssh -R 80:localhost:8080 ssh.localhost.run" will give you an internet accessible address tunneled to localhost:8080.

anderspitman 6y ago

> I set GatewayPorts clientspecified in my /etc/ssh/sshd_config on the server. This is needed so that the client (RaspberryPi) can specify which ports to enabled reverse tunneling on. This isn't quite what GatewayPorts does. By default, the client can open a tunnel to any ports the user has access to, but those ports can only be connected to from localhost on the server. So even if you don't have a firewall, external IPs can't connect to your tunnel (but your reverse proxy can, if it's running on the same machine). You can use GatewayPorts to allow binding from other IPs. This is useful if you want to tunnel arbitrary TCP traffic, not just HTTP through your reverse proxy.

aborsy 6y ago

A comparison of the security of these several approaches would be helpful. Am I correct that Wireguard and SSH are the more secure options?

anderspitman 6y ago
I would personally trust SSH based on OpenSSH the most, WireGuard second, custom SSH third, TLS 4th, and then the rest.
- aborsy 6y ago
  Interesting! I thought Wireguard is 4K lines of code while OpenSSH is hundreds of thousands of lines of code. Thus Wireguard is more secure.
  - anderspitman 6y ago
    In 5-10 years WireGuard will likely be my #1, once it's had a chance to prove itself. I have no doubt it will, but you can still hide serious security issues in 4k LOC. Especially C. Also, you can still have bugs in client implementations, and especially in the process used to pass keys around. I picked OpenSSH purely because it's been around so long and had so much data tunneled through it. That said, I would still trust WG with my data. I just don't trust it more than a proven SSH implementation, yet.

tommyvan 6y ago

A few years ago I wrote a custom SSH server to do exactly this, but for webhook development rather than tunneling to a pi. other peeps found it useful so it's turned into a bit of a side project. If you want to have a go before spending time on setting up your own server run "ssh -R 80:localhost:8080 ssh.localhost.run" to tunnel to localhost:8080, you'll see an internet accessible domain returned to your terminal.