12 ms·

The hardware is there (RPi + USB storage). The server software is there (NextCloud, Plex, n8n, etc). What isn't there is the plumbing. The next logical step aft

The hardware is there (RPi + USB storage). The server software is there (NextCloud, Plex, n8n, etc). What isn't there is the plumbing. The next logical step after this blog post is making your services accessible to your phone over the public net. You'll immediately find yourself mired in domain name registration, VPS management, TLS cert management, dyndns, port forwarding, hole punching, etc etc. There are lots of great tools that solve some of these problems. I have yet to find one that solves all of them. I think we need something like Namecheap + CloudFlare + ngrok, designed and marketed for self-hosters and federators. You simply register a domain and run a client tool on each of your machines that talks to a central server which tunnels HTTPS connections securely to the clients. Mapping X subdomain to Y port on Z machine should take a couple clicks from a web interface.

christian_fei 6y ago
I can relate, thought about setting up a Caddy server to route through the different services (also nginx would be fine). Have to try it out and probably make a list of services in a HTML document returned on port 80/443
- anderspitman 6y ago
  This is a good option if your ISP doesn't block ports 80/443 and you don't mind setting up port forwarding. EDIT: Oh and if you don't mind managing your DNS records manually, including dynamic DNS.
Aperocky 6y ago
> Mapping X subdomain to Y port on Z machine should take a couple clicks from a web interface. route53 can work like that, it also has a cli version. (But you can't get the domain there).
- anderspitman 6y ago
  Can it tunnel to local devices like a RPi or just AWS VMs?
  - Aperocky 6y ago
    You have to own the IP, and map the RPi to the standard ports (80/443, likely have to set that up from router). Alternatively just do x.com:yyyy if you don't mind (though you probably do for an external facing website).
  - duskwuff 6y ago
    No, it's just DNS. It doesn't provide any additional routing.
- an_opabinia 6y ago
  > (But you can't get the domain there) There's Amazon Domains now. Additionally, https://github.com/crazy-max/ddns-route53 https://github.com/crazy-max/ddns-route53 works well as a dynamic DNS configurator for Route 53. For most home users, a Docker-supporting server is the best option. Traefik has ACME and labels-based configuration for Docker hosts. It is a good choice for multiplexing HTTPS services by subdomain names. In my opinion the biggest limitation is that there is no universal API for network routing appliances, whether it is your $30 home combo WiFi/router or your $20,000 Cisco device. An access-key-authorized version of UPnP would be sufficient for the vast majority of users. Or even iptables commands over public key authenticated SSH. But giant corporations - Google, Microsoft, Apple, Amazon, Facebook - they are in the cloud business, Microsoft doesn't ship a home server technology really anymore. The most popular home server software, like Plex, is really purposefully disruptive to giant software and media companies. By contrast you're going to have a bad time running your own Dropbox competitor from home, because that sort of technology is engineered around cloud computing.
xyzzy_plugh 6y ago
Or just setup Tailscale, which takes about two minutes.
- anderspitman 6y ago
  Tailscale runs on WireGuard and therefore requires elevated permissions on each client device. That shouldn't be required for simply proxying a local port. Does Tailscale offer domain registration and TLS certs? Also, is there any way to allow public access to certain ports on certain machines, ie if you wanted to run your personal blog on your RPi?
  - xyzzy_plugh 6y ago
    I mean I suppose it requires elevated permissions but frankly it doesn't require any more permissions than most software, so this feels like a weird point to pick on. You need elevated permissions to bind 80 and 443, etc., right? You mentioned accessing your own devices from anywhere, and that's what I use Tailscale for. It was a dream to set up, and for my own services, I don't need TLS or custom domains, really. I have a few shortcuts on my phone that work everywhere, Tailscale IPs are static. > Also, is there any way to allow public access to certain ports on certain machines, ie if you wanted to run your personal blog on your RPi? This is sorta outside the scope of what Tailscale aims to solve, but one of the cool things you could do is just run a proxy somewhere publicly accessible and route requests to your RPi.
    - anderspitman 6y ago
      > I mean I suppose it requires elevated permissions but frankly it doesn't require any more permissions than most software, so this feels like a weird point to pick on. You need elevated permissions to bind 80 and 443, etc., right? I think maybe you're misunderstanding what my goal is. If I have a local webserver running on my laptop on port 8080, I want to expose that via HTTPS on a public domain. The server that terminates the HTTPS connection needs root to run on port 443, but my laptop doesn't need root to start the upstream webserver on 8080, and it shouldn't need root to tunnel it to the public server either.
nemothekid 6y ago
Doesn't ngrok handle all of this?
- anderspitman 6y ago
  Everything except domain registration AFAIK. Unfortunately it's closed-source and the pricing is confusing. It's not clear to me whether it would be a good choice for exposing a Plex server to your family and friends, for example.
johnjungles 6y ago
Check out KubeSail! Not affiliated in anyway. They make it super easy to do the plumbing, networking and have a kubernetes cluster on a raspberry pi. If you ever wanted to learn k8s without spending $80\month on a cluster, best way to learn it!
- erulabs 6y ago
  Thanks for the shout-out! I wanted to post "we do exactly this!" but didn't want to be an advertisement, so I appreciate it :P (co-founder of KubeSail, if anyone has any questions!)
  - anderspitman 6y ago
    Cool tech. Does KubeSail integrate domain purchasing? Why should I be required to learn kubernetes just to tunnel a local webserver to a public domain name?
    - erulabs 6y ago
      We offer free built-in (kubesail managed) domains, but don't offer domains for purchase - that would be nice to add eventually but we're a small company, so holding off on that for now! Ideally, you don't need to learn Kubernetes any more than you need to learn Linux in that example - our Repo Builder will do it's best to guess how to host your app on Kubernetes - and ideally the UI makes the rest feel like any other cloud platform. The benefit of not being locked in, and of learning open source tech instead of walled-gardens, is hard to express!
Polylactic_acid 6y ago
I tried running nextcloud on an rpi. It just doesn't cut it. I had the 4gb model and nextcloud runs but its a horrible experience. You go on the web UI and click a photo and it takes 10 seconds to load. Moved my server to a ryzen 5 based setup and now everything is instant. I'm not sure what the limiting factor on the rpi was because the ram and cpu usage was low. Perhaps it was memory or storage speed.
- salawat 6y ago
  On SD card or USB w\UASP enclosured SSD?
  - magicalhippo 6y ago
    And if USB, make sure to test the speed. Some controllers need quirks enabled[1] to get speed, including a lot of popular JMicron ones. Mine went from ~20MB/s to 300MB/s for a Samsung 850 SSD. [1]: https://www.raspberrypi.org/forums/viewtopic.php?t=245931 https://www.raspberrypi.org/forums/viewtopic.php?t=245931
  - Polylactic_acid 6y ago
    This was using an sd card.
- m463 6y ago
  I had the same experience with a pi 3 and didn't even try a pi 4. I set up a modest intel box and it was interactive. The 4 might have a better change with gigabit internet and faster usb, but I was still using a fast samsung fit drive, but not fast enough. To be honest though - the intel box was modest, but still 4x the price. Additionally it's hard to beat a pi for installation - just insert the sd card. (there is always configuration)
- guywhocodes 6y ago
  I'm doing the same, my disks are much faster than what should be necessary. Exactly the same setup and observations.
- 6y ago
z3t4 6y ago
Most home routers have functionality to do port forwarding and dyndns. Certbot for TLS/cert. Just get a free domain from the dyndns service. No VPS needed if you got your own hardware/Rasbery. Make sure you have backups. Just Dont expect a $30 PC to be without issues.
- mqus 6y ago
  For me its not the 30$ pc that has issues but the 10$ microsd card the os is on. I had to replace it for the third time in about 6 years now. But this is not something new of course, everybody will tell you to either use good uSD cards or put the os somewhere else, like an external hdd or so.
  - sorenjan 6y ago
    This is purely an anecdote, but I had to replace a few SD cards in my Pi until I changed power supply. It's not even the official, it's just one that's rated for 5.0 V, 2.5 A. I've used the same card for a few years now.
  - m-p-3 6y ago
    It's possible to boot from USB with the Pi4, I wouldn't run anything important from the MicroSD.
    - vorpalhex 6y ago
      A common setup is to pull the OS over the LAN and just use usb storage for all your actual stuff. Skip the sd card entirely except maybe as an OS fallback.
    - catdog 6y ago
      Even if direct USB boot does not work out you can just use the SD card to boot the kernel and have the root fs on USB. With the PI4 you probably can even get better performance from using a USB storage device as it has USB 3.0 Ports.
  - jsjohnst 6y ago
devwastaken 6y ago
It's against cloudflare tos to be doing mostly video/images for free. It would be a great idea as it would solve the problem of IP exposure but if it gained any traction it'd have to be shut down.
- milankragujevic 6y ago
  In my experience, you have to be way above 1 TB per day to get banned - I know because I pushed around 5000 simultaneous HD streams on my account back when I ran a pirate streaming service (Google my name if interested, it's on the Verge), and it lasted for a few days before I shut it down due to MPAA, but didn't get banned by CloudFlare. I still have the account that I did that on, it's from 2011. and still used for a few of my sites.
cguess 6y ago
> I think we need something like Namecheap + CloudFlare + ngrok, designed and marketed for self-hosters and federators. You simply register a domain and run a client tool on each of your machines that talks to a central server which tunnels HTTPS connections securely to the clients. PageKite (https://www.pagekite.net https://www.pagekite.net) is what it sounds like you're looking for. It'll set you up with a url, SSL, and a tunnel in about 30 seconds. Highly configurable for your own domain if you'd like, multiple ports etc.
- tomovo 6y ago
  That PageKite pricing slider is really great. https://pagekite.net/signup/?more=bw https://pagekite.net/signup/?more=bw
  - sokoloff 6y ago
    It looks like it could be good, but doesn’t slide on Safari on my iPhone.
- sairamkunala 6y ago
  and there is inlets too https://github.com/inlets/inletsctl https://github.com/inlets/inletsctl
chrios 6y ago
Use IPv6. It doesn't have the address exhaustion that caused providers to implement CGNAT, and dynamic IPv4 addresses. No need for VPS management, DynDNS, port forwarding, hold punching. You still need public DNS, but you can use public DNS as your internal zone as well (no need for split DNS). You also still need PKI, so maybe setup a reverse proxy for SSL termination with a wildcard certificate.
- d1plo1d 6y ago
  I look forward to the day everyone can have IPV6. For now many of us still have to deal with NAT sadly - especially if we're open sourcing our software for users to deploy on any network.
  - LargoLasskhyfv 6y ago
    Everyone can have IPv6 today by using a tunnelbroker. I used the free tunnel from https://www.he.net/ https://www.he.net/ in the past, when I didn't have native v6. Today I don't need it anymore.
    - d1plo1d 6y ago
      Wow, am I getting this right? It handles NAT traversal for you behind the IPV6 address for free??
      - LargoLasskhyfv 6y ago
        What do you mean by that exactly? Initially it's just an outgoing tunnel to one of their many exits, to reach any site which is reachable via v6. How you integrate that into your setup is up to you. Since they are (one of?) the pioneers you have many scripts available on many platforms which support that. When you mean incoming tunnel, it's no different from the many dynamic DNS solutions, where it's again up to you to integrate that. But even for this they have something: https://dns.he.net/ https://dns.he.net/
        6y ago
coronadisaster 6y ago
I use a VPN + static IP or DDNS to get access to my home cloud/server (both the VPN and DDNS can be setup on my router). Also, there are free DDNS providers.
d1plo1d 6y ago
I tunnel everything through webRTC. It's a bit exotic but it gets you a direct bidirectional data connection to the self hosted device. You can put all users' self hosted content through a single domain name & SSL cert or you could have subdomains automatically provisioned for each device. I'm using this WebRTC method for 3D printers at https://tegapp.io https://tegapp.io
- anderspitman 6y ago
  Can you provide more details on what software you're using for WebRTC tunneling?
  - d1plo1d 6y ago
    Sure, right now I'm using a nodejs WebRTC datachannels implementation but there's an up and coming rust implementation which I'm quite excited to try: - NodeJS DataChannels: https://github.com/node-webrtc/node-webrtc https://github.com/node-webrtc/node-webrtc - Rust DataChannels: https://github.com/lerouxrgd/datachannel-rs https://github.com/lerouxrgd/datachannel-rs
homedrive 6y ago
HomeDrive ( https://www.homedrive.io https://www.homedrive.io ) is plumbing exactly this! We are currently only hosting Nextcloud, but we plan to support more apps and custom dockers. It is as easy as plugging the box into the home router. There are still many features to implement, but we are working towards "easy self-hosting at home", and looking for early adopters.
- anderspitman 6y ago
  How would you compare yourselves to KubeSail, mentioned above? Why limit it to specific software, rather than simply port mapping?
  - homedrive 6y ago
    HomeDrive features plug-and-use, with no system maintenance required. We target not only developers/hackers, but also end-users who would like to have a small server at home, to host their own data and services for their digital lives. As a result, HomeDrive also maintains the OS to be reliable and secure, which is why we picked the hardware. We are investigating supporting more hardware, such as raspberry pi's. KubeSail feels like a more accessible k8s+docker apps to me. I am not sure what security model KubeSail is assuming for the operating systems it is running on (or does it assume the OS is out of the scope?). Also KubeSail seems to target mostly developers/hackers.
m-p-3 6y ago
Caddy? https://caddyserver.com/ https://caddyserver.com/
- anderspitman 6y ago
  Caddy is great, and it'll take care of managing the TLS certs. There's a lot left on my wishlist above...
wpearse 6y ago
ZeroTier might solve some of this.
angrygoat 6y ago
I've got IPv6 from my ISP - I gave my pi a static v6 IP, set up cloudflare and told it that v6 IP, and cloudflare handles making it available to the v4 internet. Has worked pretty well so far, just for tinkering stuff, nothing 'production'.
gramakri 6y ago
https://cloudron.io https://cloudron.io can do most of this minus the hole punching . The port forward ing is very router specific. I think maybe there is some upnp interface for this but not sure how widely it is supported.
colechristensen 6y ago
With a couple of fiber providers I've been lucky enough (it wasn't luck, I chose housing based on ISP availability) to get a business class gigabit with a static ipv4 address and ipv6 for ~$100/mo, solves lots of problems. Plex does the plumbing for you, I think NextCloud might too. Doing DNS just seems like another thing to setup which is just fine. Pay namecheap, then pay a big boy DNS provider (I like dnsmadeeasy) then register some domains. I don't really want a solves everything tool because I don't see a way for it not to be really opinionated and hide everything behind its own abstraction which isn't really any better than the interface it is hiding. Maybe a series of how-to whitepaper kinds of thing to build up the requisite knowledge to figure these things out. I'm not a fan of the old school configuration hell where you have to spend hours/days/weeks trying to figure out the correct set of software and config options to do something right, but I'm equally not a fan of completely canned solutions that hide everything in favor of a single button to push. I'm not a technician, I don't need to have everything done for me, but I do appreciate tools where right configuration interface is provided. That is, sane defaults, well documented options, meaningful errors and sanity checking, and options given in the right way.
- alexktz 6y ago
  This is exactly why containers took off for self-hosters. Installing Plex went from a blog post to `docker run plex` (ok there is a little more nuance than that but you get my point). Docker allowed me to be far more competent on a Linux box than my skill set should have permitted at the time. I no longer needed to know how an application ran, just that it did. Provide some persistent storage and you’ll probably never have to configure that app again. Amazing.
  - colechristensen 6y ago
    But if you have a package manager, 'yum install plex', 'apt install plex', etc. should all have the same experience. (actually the correct incantation of docker to get plex running correctly has taken hours of my life away). There are indeed many blog posts about getting plex to run in docker. The problem is package managers are bad (well, apt is bad, rpm is pretty ok, freebsd ports are pretty good, there are many others) and package maintainers are bad, it always seems like the job they give the intern to figure out instead of making it a cornerstone of usability. Seriously, spend a day setting up things running on freebsd with the packaging you have and it will be a breath of fresh air. Nearly everything you can think of all put together in packages in one place, and most of them start working in an expected way with zero configuration fiddling, install and start. If package management were better, docker might not have existed at all, people keep confusing it with decent package management.
RealStickman_ 6y ago
I'm currently thinking of using a reverse proxy through a wireguard tunnel. That should work also for non-static home ip addresses. (I already habe the domain and VPS)
bashallah 6y ago
I need this. Bought a namecheap registry for a small nfp. Been swamped with how-to’s and learning things just to learn what to search for... I could use something else that does it all... But I want a level of authority none of those offer.. without the technical insight of “is this everything/enough”.
eat_veggies 6y ago
Cloudflare argo tunnels are exactly what you're looking for https://developers.cloudflare.com/argo-tunnel/quickstart https://developers.cloudflare.com/argo-tunnel/quickstart
- anderspitman 6y ago
  I'm aware of argo tunnels. Unfortunately: * Argo smart routing is 5 USD/mo + 0.1 USD/GB. The 5/mo is fine, but the data charges could add up quickly for something like Plex. * CloudFlare doesn't sell domains.
  - sdfhbdf 6y ago
    > CloudFlare doesn't sell domains They do have a domain registrar intermediary [1] announced two years ago [2]. It's in cooperation with dount.domains and has competetive pricing. It could be counted as they do sell domains. Not affiliated with them in any way. [1]: https://www.cloudflare.com/products/registrar/ https://www.cloudflare.com/products/registrar/ [2]: https://blog.cloudflare.com/cloudflare-registrar/ https://blog.cloudflare.com/cloudflare-registrar/
    - anderspitman 6y ago
      Looks like you can only port domains you already have. Announced 2 years ago and still no general availability isn't a great sign.
jeroenhd 6y ago
There's a lot of solutions sibling comments have already brought up, but I don't know if it should be this automagical. Keeping services up to date requires effort, money, or a big reduction in freedom of what you can do with your server. There's a full-automatic mail server program, maininabox, that tries to be this instant "just make it work" system. The result of the project is that the host OS was severely outdated for years because upgrading configuration automatically is difficult and because the system manages DNS for you, adding a new subdomein to your server is more of a challenge than it should be. Similarly, automatic service install and management tools like Plesk, cPanel, ISPconfig have been around forever but they always provide some limitation. I think Sandatorm.IO is a quite recent tool of this sort that runs Docker so you have a bit more control. All of these still require occasional maintenance though. If you can't figure out how to point a DNS name and a wildcard to your IP, then I'm not sure if you should be exposing services on the internet like that. If you don't update for a while your nice, powerful server Raspberry Pi might suddenly be DDOS'ing random websites without you even knowing about it, and all you can do to prevent that is to keep your (limited) software stack updated. All attempts to make this easy for the general public have so far shown that people don't like to press the update button; even rebooting Windows is a risk some people just aren't willing to take, which is why Microsoft had to force reboots in Windows 10. With that kind of risk out there, freely connecting whatever to the web and forgetting about it, I'm glad there's some technical requirements before you can host something.
- troyjfarrell 6y ago
  Sandstorm(.io) is very cool, and it does make managing your self-hosted web apps very easy. But it does not run Docker containers and it only runs on Linux x86-64. (There have been some attempts at running Docker containers with Sandstorm, but they are not easy to use.) Instead, the web applications must be specifically packaged for Sandstorm.
  - jeroenhd 6y ago
    Oh, I suppose I was mistaken. Perhaps I confused it with one of its competitors I can't remember the name of right now.
phaer 6y ago
> The next logical step after this blog post is making your services accessible to your phone over the public net. You'll immediately find yourself mired in domain name registration, VPS management, TLS cert management, dyndns, port forwarding, hole punching, etc etc. You'll need don't need any of that with Onion Services. Tor does not only anonymize, but offers easily configurable services with NAT punching, an .onion Domain and e2e crypto for free. And setting them up is easy enough https://community.torproject.org/onion-services/setup/ https://community.torproject.org/onion-services/setup/ You'll just need tor or a tor browser to access those services, but that shouldn't be a problem for many self-hosting setups
- polytely 6y ago
  Wow that is interesting, never thought about this usecase for Tor before, that looks like a fun project to figure out
- fareesh 6y ago
  Isn't tor exceptionally slow though? I haven't used it in a few years - has anything changed?
  - krageon 6y ago
    It is. It's not practical for this use-case.
  - aborsy 6y ago
    It’s slow, with NSA hunting at exit nodes.
    - phaer 6y ago
      Exit nodes aren't used if one connects to an onion service. And for all we know, the NSA snoops traffic at all major internet exchanges, so tor exit nodes might get extra attention, but so do e.g. people who's search history suggests they might be sysadmins (if i remember reports on xkeyscore selector correctly)
  - 6y ago
Jnr 6y ago
For internal only things you can use Wireguard VPN and any dynamic DNS provider.
wwn_se 6y ago
Cloudflare -> Router (only allows 80 443 trafic from cloudflare ips) -> nginx -> all selfhosted services (wiki, hass...) Problems with the "easy" one click ones is that they tend to not be very secure. If they are supposed to be public access. Plex uses their cloud to secure the access and Synology to
cik 6y ago
Start by having a look at subspace https://github.com/subspacecloud/subspace https://github.com/subspacecloud/subspace
- jeroenhd 6y ago
  The original repo is receiving very little updates and maintenance, if any. I'd recommend the community fork at https://github.com/subspacecommunity/subspace https://github.com/subspacecommunity/subspace instead.
intricatedetail 6y ago
You can run zerotier and have your services on your own private network accessible from anywhere.
anderspitman 6y ago
I've been going down the rabbit hole looking through different software in this space. I started an awesome-list to track what I've learned: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling
shifto 6y ago
> Mapping X subdomain to Y port on Z machine should take a couple clicks from a web interface. This is already the case. Routers have mostly easy webinterfaces nowadays en the same goes for DNS options at any domainname provider. What people need is a bit of knowledge. It takes me a few mouse clicks in a webinterface to do this because I know what I am doing. Yeah you could dumb down anything to a single button but I don't think we should want that.

christian_fei 6y ago

I can relate, thought about setting up a Caddy server to route through the different services (also nginx would be fine). Have to try it out and probably make a list of services in a HTML document returned on port 80/443

anderspitman 6y ago
This is a good option if your ISP doesn't block ports 80/443 and you don't mind setting up port forwarding. EDIT: Oh and if you don't mind managing your DNS records manually, including dynamic DNS.

Aperocky 6y ago

> Mapping X subdomain to Y port on Z machine should take a couple clicks from a web interface. route53 can work like that, it also has a cli version. (But you can't get the domain there).

anderspitman 6y ago
Can it tunnel to local devices like a RPi or just AWS VMs?
- Aperocky 6y ago
  You have to own the IP, and map the RPi to the standard ports (80/443, likely have to set that up from router). Alternatively just do x.com:yyyy if you don't mind (though you probably do for an external facing website).
- duskwuff 6y ago
  No, it's just DNS. It doesn't provide any additional routing.
an_opabinia 6y ago
> (But you can't get the domain there) There's Amazon Domains now. Additionally, https://github.com/crazy-max/ddns-route53 https://github.com/crazy-max/ddns-route53 works well as a dynamic DNS configurator for Route 53. For most home users, a Docker-supporting server is the best option. Traefik has ACME and labels-based configuration for Docker hosts. It is a good choice for multiplexing HTTPS services by subdomain names. In my opinion the biggest limitation is that there is no universal API for network routing appliances, whether it is your $30 home combo WiFi/router or your $20,000 Cisco device. An access-key-authorized version of UPnP would be sufficient for the vast majority of users. Or even iptables commands over public key authenticated SSH. But giant corporations - Google, Microsoft, Apple, Amazon, Facebook - they are in the cloud business, Microsoft doesn't ship a home server technology really anymore. The most popular home server software, like Plex, is really purposefully disruptive to giant software and media companies. By contrast you're going to have a bad time running your own Dropbox competitor from home, because that sort of technology is engineered around cloud computing.

xyzzy_plugh 6y ago

Or just setup Tailscale, which takes about two minutes.

anderspitman 6y ago
Tailscale runs on WireGuard and therefore requires elevated permissions on each client device. That shouldn't be required for simply proxying a local port. Does Tailscale offer domain registration and TLS certs? Also, is there any way to allow public access to certain ports on certain machines, ie if you wanted to run your personal blog on your RPi?
- xyzzy_plugh 6y ago
  I mean I suppose it requires elevated permissions but frankly it doesn't require any more permissions than most software, so this feels like a weird point to pick on. You need elevated permissions to bind 80 and 443, etc., right? You mentioned accessing your own devices from anywhere, and that's what I use Tailscale for. It was a dream to set up, and for my own services, I don't need TLS or custom domains, really. I have a few shortcuts on my phone that work everywhere, Tailscale IPs are static. > Also, is there any way to allow public access to certain ports on certain machines, ie if you wanted to run your personal blog on your RPi? This is sorta outside the scope of what Tailscale aims to solve, but one of the cool things you could do is just run a proxy somewhere publicly accessible and route requests to your RPi.
  - anderspitman 6y ago
    > I mean I suppose it requires elevated permissions but frankly it doesn't require any more permissions than most software, so this feels like a weird point to pick on. You need elevated permissions to bind 80 and 443, etc., right? I think maybe you're misunderstanding what my goal is. If I have a local webserver running on my laptop on port 8080, I want to expose that via HTTPS on a public domain. The server that terminates the HTTPS connection needs root to run on port 443, but my laptop doesn't need root to start the upstream webserver on 8080, and it shouldn't need root to tunnel it to the public server either.

nemothekid 6y ago

Doesn't ngrok handle all of this?

anderspitman 6y ago
Everything except domain registration AFAIK. Unfortunately it's closed-source and the pricing is confusing. It's not clear to me whether it would be a good choice for exposing a Plex server to your family and friends, for example.

johnjungles 6y ago

Check out KubeSail! Not affiliated in anyway. They make it super easy to do the plumbing, networking and have a kubernetes cluster on a raspberry pi. If you ever wanted to learn k8s without spending $80\month on a cluster, best way to learn it!

erulabs 6y ago
Thanks for the shout-out! I wanted to post "we do exactly this!" but didn't want to be an advertisement, so I appreciate it :P (co-founder of KubeSail, if anyone has any questions!)
- anderspitman 6y ago
  Cool tech. Does KubeSail integrate domain purchasing? Why should I be required to learn kubernetes just to tunnel a local webserver to a public domain name?
  - erulabs 6y ago
    We offer free built-in (kubesail managed) domains, but don't offer domains for purchase - that would be nice to add eventually but we're a small company, so holding off on that for now! Ideally, you don't need to learn Kubernetes any more than you need to learn Linux in that example - our Repo Builder will do it's best to guess how to host your app on Kubernetes - and ideally the UI makes the rest feel like any other cloud platform. The benefit of not being locked in, and of learning open source tech instead of walled-gardens, is hard to express!

Polylactic_acid 6y ago

I tried running nextcloud on an rpi. It just doesn't cut it. I had the 4gb model and nextcloud runs but its a horrible experience. You go on the web UI and click a photo and it takes 10 seconds to load. Moved my server to a ryzen 5 based setup and now everything is instant. I'm not sure what the limiting factor on the rpi was because the ram and cpu usage was low. Perhaps it was memory or storage speed.

salawat 6y ago
On SD card or USB w\UASP enclosured SSD?
- magicalhippo 6y ago
  And if USB, make sure to test the speed. Some controllers need quirks enabled[1] to get speed, including a lot of popular JMicron ones. Mine went from ~20MB/s to 300MB/s for a Samsung 850 SSD. [1]: https://www.raspberrypi.org/forums/viewtopic.php?t=245931 https://www.raspberrypi.org/forums/viewtopic.php?t=245931
- Polylactic_acid 6y ago
  This was using an sd card.
m463 6y ago
I had the same experience with a pi 3 and didn't even try a pi 4. I set up a modest intel box and it was interactive. The 4 might have a better change with gigabit internet and faster usb, but I was still using a fast samsung fit drive, but not fast enough. To be honest though - the intel box was modest, but still 4x the price. Additionally it's hard to beat a pi for installation - just insert the sd card. (there is always configuration)
guywhocodes 6y ago
I'm doing the same, my disks are much faster than what should be necessary. Exactly the same setup and observations.
6y ago

z3t4 6y ago

Most home routers have functionality to do port forwarding and dyndns. Certbot for TLS/cert. Just get a free domain from the dyndns service. No VPS needed if you got your own hardware/Rasbery. Make sure you have backups. Just Dont expect a $30 PC to be without issues.

mqus 6y ago
For me its not the 30$ pc that has issues but the 10$ microsd card the os is on. I had to replace it for the third time in about 6 years now. But this is not something new of course, everybody will tell you to either use good uSD cards or put the os somewhere else, like an external hdd or so.
- sorenjan 6y ago
  This is purely an anecdote, but I had to replace a few SD cards in my Pi until I changed power supply. It's not even the official, it's just one that's rated for 5.0 V, 2.5 A. I've used the same card for a few years now.
- m-p-3 6y ago
  It's possible to boot from USB with the Pi4, I wouldn't run anything important from the MicroSD.
  - vorpalhex 6y ago
    A common setup is to pull the OS over the LAN and just use usb storage for all your actual stuff. Skip the sd card entirely except maybe as an OS fallback.
  - catdog 6y ago
    Even if direct USB boot does not work out you can just use the SD card to boot the kernel and have the root fs on USB. With the PI4 you probably can even get better performance from using a USB storage device as it has USB 3.0 Ports.
- jsjohnst 6y ago

devwastaken 6y ago

It's against cloudflare tos to be doing mostly video/images for free. It would be a great idea as it would solve the problem of IP exposure but if it gained any traction it'd have to be shut down.

milankragujevic 6y ago
In my experience, you have to be way above 1 TB per day to get banned - I know because I pushed around 5000 simultaneous HD streams on my account back when I ran a pirate streaming service (Google my name if interested, it's on the Verge), and it lasted for a few days before I shut it down due to MPAA, but didn't get banned by CloudFlare. I still have the account that I did that on, it's from 2011. and still used for a few of my sites.

cguess 6y ago

> I think we need something like Namecheap + CloudFlare + ngrok, designed and marketed for self-hosters and federators. You simply register a domain and run a client tool on each of your machines that talks to a central server which tunnels HTTPS connections securely to the clients. PageKite (https://www.pagekite.net https://www.pagekite.net) is what it sounds like you're looking for. It'll set you up with a url, SSL, and a tunnel in about 30 seconds. Highly configurable for your own domain if you'd like, multiple ports etc.

tomovo 6y ago
That PageKite pricing slider is really great. https://pagekite.net/signup/?more=bw https://pagekite.net/signup/?more=bw
- sokoloff 6y ago
  It looks like it could be good, but doesn’t slide on Safari on my iPhone.
sairamkunala 6y ago
and there is inlets too https://github.com/inlets/inletsctl https://github.com/inlets/inletsctl

chrios 6y ago

Use IPv6. It doesn't have the address exhaustion that caused providers to implement CGNAT, and dynamic IPv4 addresses. No need for VPS management, DynDNS, port forwarding, hold punching. You still need public DNS, but you can use public DNS as your internal zone as well (no need for split DNS). You also still need PKI, so maybe setup a reverse proxy for SSL termination with a wildcard certificate.

d1plo1d 6y ago
I look forward to the day everyone can have IPV6. For now many of us still have to deal with NAT sadly - especially if we're open sourcing our software for users to deploy on any network.
- LargoLasskhyfv 6y ago
  Everyone can have IPv6 today by using a tunnelbroker. I used the free tunnel from https://www.he.net/ https://www.he.net/ in the past, when I didn't have native v6. Today I don't need it anymore.
  - d1plo1d 6y ago
    Wow, am I getting this right? It handles NAT traversal for you behind the IPV6 address for free??
    - LargoLasskhyfv 6y ago
      What do you mean by that exactly? Initially it's just an outgoing tunnel to one of their many exits, to reach any site which is reachable via v6. How you integrate that into your setup is up to you. Since they are (one of?) the pioneers you have many scripts available on many platforms which support that. When you mean incoming tunnel, it's no different from the many dynamic DNS solutions, where it's again up to you to integrate that. But even for this they have something: https://dns.he.net/ https://dns.he.net/
      - 6y ago

coronadisaster 6y ago

I use a VPN + static IP or DDNS to get access to my home cloud/server (both the VPN and DDNS can be setup on my router). Also, there are free DDNS providers.

d1plo1d 6y ago

I tunnel everything through webRTC. It's a bit exotic but it gets you a direct bidirectional data connection to the self hosted device. You can put all users' self hosted content through a single domain name & SSL cert or you could have subdomains automatically provisioned for each device. I'm using this WebRTC method for 3D printers at https://tegapp.io https://tegapp.io

anderspitman 6y ago
Can you provide more details on what software you're using for WebRTC tunneling?
- d1plo1d 6y ago
  Sure, right now I'm using a nodejs WebRTC datachannels implementation but there's an up and coming rust implementation which I'm quite excited to try: - NodeJS DataChannels: https://github.com/node-webrtc/node-webrtc https://github.com/node-webrtc/node-webrtc - Rust DataChannels: https://github.com/lerouxrgd/datachannel-rs https://github.com/lerouxrgd/datachannel-rs

homedrive 6y ago

HomeDrive ( https://www.homedrive.io https://www.homedrive.io ) is plumbing exactly this! We are currently only hosting Nextcloud, but we plan to support more apps and custom dockers. It is as easy as plugging the box into the home router. There are still many features to implement, but we are working towards "easy self-hosting at home", and looking for early adopters.

anderspitman 6y ago
How would you compare yourselves to KubeSail, mentioned above? Why limit it to specific software, rather than simply port mapping?
- homedrive 6y ago
  HomeDrive features plug-and-use, with no system maintenance required. We target not only developers/hackers, but also end-users who would like to have a small server at home, to host their own data and services for their digital lives. As a result, HomeDrive also maintains the OS to be reliable and secure, which is why we picked the hardware. We are investigating supporting more hardware, such as raspberry pi's. KubeSail feels like a more accessible k8s+docker apps to me. I am not sure what security model KubeSail is assuming for the operating systems it is running on (or does it assume the OS is out of the scope?). Also KubeSail seems to target mostly developers/hackers.

m-p-3 6y ago

Caddy? https://caddyserver.com/ https://caddyserver.com/

anderspitman 6y ago
Caddy is great, and it'll take care of managing the TLS certs. There's a lot left on my wishlist above...

wpearse 6y ago

ZeroTier might solve some of this.

angrygoat 6y ago

I've got IPv6 from my ISP - I gave my pi a static v6 IP, set up cloudflare and told it that v6 IP, and cloudflare handles making it available to the v4 internet. Has worked pretty well so far, just for tinkering stuff, nothing 'production'.

gramakri 6y ago

https://cloudron.io https://cloudron.io can do most of this minus the hole punching . The port forward ing is very router specific. I think maybe there is some upnp interface for this but not sure how widely it is supported.

colechristensen 6y ago

With a couple of fiber providers I've been lucky enough (it wasn't luck, I chose housing based on ISP availability) to get a business class gigabit with a static ipv4 address and ipv6 for ~$100/mo, solves lots of problems. Plex does the plumbing for you, I think NextCloud might too. Doing DNS just seems like another thing to setup which is just fine. Pay namecheap, then pay a big boy DNS provider (I like dnsmadeeasy) then register some domains. I don't really want a solves everything tool because I don't see a way for it not to be really opinionated and hide everything behind its own abstraction which isn't really any better than the interface it is hiding. Maybe a series of how-to whitepaper kinds of thing to build up the requisite knowledge to figure these things out. I'm not a fan of the old school configuration hell where you have to spend hours/days/weeks trying to figure out the correct set of software and config options to do something right, but I'm equally not a fan of completely canned solutions that hide everything in favor of a single button to push. I'm not a technician, I don't need to have everything done for me, but I do appreciate tools where right configuration interface is provided. That is, sane defaults, well documented options, meaningful errors and sanity checking, and options given in the right way.

alexktz 6y ago
This is exactly why containers took off for self-hosters. Installing Plex went from a blog post to `docker run plex` (ok there is a little more nuance than that but you get my point). Docker allowed me to be far more competent on a Linux box than my skill set should have permitted at the time. I no longer needed to know how an application ran, just that it did. Provide some persistent storage and you’ll probably never have to configure that app again. Amazing.
- colechristensen 6y ago
  But if you have a package manager, 'yum install plex', 'apt install plex', etc. should all have the same experience. (actually the correct incantation of docker to get plex running correctly has taken hours of my life away). There are indeed many blog posts about getting plex to run in docker. The problem is package managers are bad (well, apt is bad, rpm is pretty ok, freebsd ports are pretty good, there are many others) and package maintainers are bad, it always seems like the job they give the intern to figure out instead of making it a cornerstone of usability. Seriously, spend a day setting up things running on freebsd with the packaging you have and it will be a breath of fresh air. Nearly everything you can think of all put together in packages in one place, and most of them start working in an expected way with zero configuration fiddling, install and start. If package management were better, docker might not have existed at all, people keep confusing it with decent package management.

RealStickman_ 6y ago

I'm currently thinking of using a reverse proxy through a wireguard tunnel. That should work also for non-static home ip addresses. (I already habe the domain and VPS)

bashallah 6y ago

I need this. Bought a namecheap registry for a small nfp. Been swamped with how-to’s and learning things just to learn what to search for... I could use something else that does it all... But I want a level of authority none of those offer.. without the technical insight of “is this everything/enough”.

eat_veggies 6y ago

Cloudflare argo tunnels are exactly what you're looking for https://developers.cloudflare.com/argo-tunnel/quickstart https://developers.cloudflare.com/argo-tunnel/quickstart

anderspitman 6y ago
I'm aware of argo tunnels. Unfortunately: * Argo smart routing is 5 USD/mo + 0.1 USD/GB. The 5/mo is fine, but the data charges could add up quickly for something like Plex. * CloudFlare doesn't sell domains.
- sdfhbdf 6y ago
  > CloudFlare doesn't sell domains They do have a domain registrar intermediary [1] announced two years ago [2]. It's in cooperation with dount.domains and has competetive pricing. It could be counted as they do sell domains. Not affiliated with them in any way. [1]: https://www.cloudflare.com/products/registrar/ https://www.cloudflare.com/products/registrar/ [2]: https://blog.cloudflare.com/cloudflare-registrar/ https://blog.cloudflare.com/cloudflare-registrar/
  - anderspitman 6y ago
    Looks like you can only port domains you already have. Announced 2 years ago and still no general availability isn't a great sign.

jeroenhd 6y ago

There's a lot of solutions sibling comments have already brought up, but I don't know if it should be this automagical. Keeping services up to date requires effort, money, or a big reduction in freedom of what you can do with your server. There's a full-automatic mail server program, maininabox, that tries to be this instant "just make it work" system. The result of the project is that the host OS was severely outdated for years because upgrading configuration automatically is difficult and because the system manages DNS for you, adding a new subdomein to your server is more of a challenge than it should be. Similarly, automatic service install and management tools like Plesk, cPanel, ISPconfig have been around forever but they always provide some limitation. I think Sandatorm.IO is a quite recent tool of this sort that runs Docker so you have a bit more control. All of these still require occasional maintenance though. If you can't figure out how to point a DNS name and a wildcard to your IP, then I'm not sure if you should be exposing services on the internet like that. If you don't update for a while your nice, powerful server Raspberry Pi might suddenly be DDOS'ing random websites without you even knowing about it, and all you can do to prevent that is to keep your (limited) software stack updated. All attempts to make this easy for the general public have so far shown that people don't like to press the update button; even rebooting Windows is a risk some people just aren't willing to take, which is why Microsoft had to force reboots in Windows 10. With that kind of risk out there, freely connecting whatever to the web and forgetting about it, I'm glad there's some technical requirements before you can host something.

troyjfarrell 6y ago
Sandstorm(.io) is very cool, and it does make managing your self-hosted web apps very easy. But it does not run Docker containers and it only runs on Linux x86-64. (There have been some attempts at running Docker containers with Sandstorm, but they are not easy to use.) Instead, the web applications must be specifically packaged for Sandstorm.
- jeroenhd 6y ago
  Oh, I suppose I was mistaken. Perhaps I confused it with one of its competitors I can't remember the name of right now.

phaer 6y ago

> The next logical step after this blog post is making your services accessible to your phone over the public net. You'll immediately find yourself mired in domain name registration, VPS management, TLS cert management, dyndns, port forwarding, hole punching, etc etc. You'll need don't need any of that with Onion Services. Tor does not only anonymize, but offers easily configurable services with NAT punching, an .onion Domain and e2e crypto for free. And setting them up is easy enough https://community.torproject.org/onion-services/setup/ https://community.torproject.org/onion-services/setup/ You'll just need tor or a tor browser to access those services, but that shouldn't be a problem for many self-hosting setups

polytely 6y ago
Wow that is interesting, never thought about this usecase for Tor before, that looks like a fun project to figure out
fareesh 6y ago
Isn't tor exceptionally slow though? I haven't used it in a few years - has anything changed?
- krageon 6y ago
  It is. It's not practical for this use-case.
- aborsy 6y ago
  It’s slow, with NSA hunting at exit nodes.
  - phaer 6y ago
    Exit nodes aren't used if one connects to an onion service. And for all we know, the NSA snoops traffic at all major internet exchanges, so tor exit nodes might get extra attention, but so do e.g. people who's search history suggests they might be sysadmins (if i remember reports on xkeyscore selector correctly)
- 6y ago

Jnr 6y ago

For internal only things you can use Wireguard VPN and any dynamic DNS provider.

wwn_se 6y ago

Cloudflare -> Router (only allows 80 443 trafic from cloudflare ips) -> nginx -> all selfhosted services (wiki, hass...) Problems with the "easy" one click ones is that they tend to not be very secure. If they are supposed to be public access. Plex uses their cloud to secure the access and Synology to

cik 6y ago

Start by having a look at subspace https://github.com/subspacecloud/subspace https://github.com/subspacecloud/subspace

jeroenhd 6y ago
The original repo is receiving very little updates and maintenance, if any. I'd recommend the community fork at https://github.com/subspacecommunity/subspace https://github.com/subspacecommunity/subspace instead.

intricatedetail 6y ago

You can run zerotier and have your services on your own private network accessible from anywhere.

anderspitman 6y ago

I've been going down the rabbit hole looking through different software in this space. I started an awesome-list to track what I've learned: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling

shifto 6y ago

> Mapping X subdomain to Y port on Z machine should take a couple clicks from a web interface. This is already the case. Routers have mostly easy webinterfaces nowadays en the same goes for DNS options at any domainname provider. What people need is a bit of knowledge. It takes me a few mouse clicks in a webinterface to do this because I know what I am doing. Yeah you could dumb down anything to a single button but I don't think we should want that.