10 ms·

A guide for getting started with self hosting

143 points by wagslane 4y ago

wrycoder 4y ago
Maybe I missed it, but I don't see anything about discoverability. That's a big problem for servers on residential IP ranges.
- abriosi 4y ago
  This is one of the challenges I have faced with russing servers on local IPs. Do you have any resources you can share about this subject?
  - woxerrat 4y ago
    https://freemyip.com/ https://freemyip.com/ is a good tool when on residential network with no static ip from ISP.
    - feet 4y ago
      Or you can rent a vps for $5 per month and wireguard your internal services to the vps. Vps gets put in the DNS records and throw traefik or caddy on the vps to handle things
      - squarefoot 4y ago
        Some HN user months ago suggested the entry level VPS offer from ionos.com which is $2/month at the moment. I bookmarked the link because I will probably need it when I relocate, but never used them so far. Any comments from customers are welcome. https://www.ionos.com/servers/vps#plans https://www.ionos.com/servers/vps#plans
- jve 4y ago
  I'd also like to learn more about about this subject. Cloudflare offers some API to update DNS: https://developers.cloudflare.com/dns/manage-dns-records/how-to/managing-dynamic-ip-addresses/ https://developers.cloudflare.com/dns/manage-dns-records/how... I'd like to see some of those scripts that use CF API to detect and update IP in dynamics public IP environment.
  - thedanbob 4y ago
    It’s not particularly pretty, but here is my home-grown script for doing exactly that. It coordinates between Cloudflare DNS, my Unifi Security Gateway, and haproxy running on my home servers to keep my IPv4 and IPv6 addresses up to date everywhere. https://gist.github.com/thedanbob/df4cc49aa3e60581e010f443acfe254e https://gist.github.com/thedanbob/df4cc49aa3e60581e010f443ac...
  - jcynix 4y ago
    If you have a server with a public address, a machine with a private address can simply use e.g. curl to access some (even non-existant) url on the server periodically which can then be extracted from the logs by a script on the server. I actually use this to announce my internal address, even without a web server, just by accessing a locked port on the server, where the firewall logs this access.
  - watchdogtimer 4y ago
    I used to use Cloudflare's API to do this, but then we changed ISPs and now are behind CGNAT, so we don't even have a public IP. To publicly expose my web server, I created a Cloudflared tunnel, and have our DNS (which is also hosted at Cloudfare) point to the tunnel. It works quite well, and adds some security because we're no longer visible to attackers using IP address scans.
- deniska 4y ago
  I have a small script ran by cron (well, systemd timers) every 15 minutes which checks the current IP via https://api.ipify.org/ https://api.ipify.org/ and updates dns entries on digital ocean (its dns service is free) if it differs from what's there.
  - KronisLV 4y ago
    If someone has to deal with dynamic IP addresses and wants to use dynamic DNS as well, there's also the pretty nice ddclient package: https://ddclient.net/ https://ddclient.net/ I've been using it with NameCheap pretty successfully for a while now, here's an example of some of the integrations it supports: https://ddclient.net/protocols.html https://ddclient.net/protocols.html (sadly no DigitalOcean integration as of yet, but this might be of use to others)
- cassianoleal 4y ago
  Self-hosting means hosting things yourself. If you don't know what you host that's a problem. Otherwise, discovery is largely irrelevant and residential IPs have no bearing on it. For actual reachability, a lot of people use VPNs, network meshes (Tailscale, ZeroTier, Nebula, etc) or secure proxies (Cloudflare Argo Tunnels). You can also self-host software on VPSs or cloud servers.
- superkuh 4y ago
  Discoverability? Uh... I think you may be dragging commercial concepts into this that don't have a place here. For a personal website people discover it when you link them to it. Or discover it 4 years later after google has indexed it and it happens organically. But you don't ever have to care about "discoverability". You're not trying to get a bunch of hits, or profit, or whatever. A personal website is something you do for fun. If this personal website is serving as your resume to get hired or sell stuff, it's not a personal website. It's more commercial/profit crap. At that point, yeah, use all the cargo cult you want. Business want to see the tools and technologies they use demonstrated. I've been hosting superkuh.com from my home PC for 20 years. When my IP changes (maybe once per year) I just log into my registrar when I notice it (it's okay if I don't notice for a day or two, no big deal) and update it.
dopidopHN 4y ago
No yunohost ?
readingnews 4y ago
Maybe I missed it, but this just looks like a giant list of open source tools. Docker, Containers, virtualization, password management, etc. Not knocking OP, but a "guide for getting started" seems like it would be a howto for self hosting. This seems like a giant list of tools. I only note this as I work at a place that self-hosts everything as we already have an on-prem data center. For example, we use openstack for our users to spin up VMs. If I did a guide to self hosting, I would talk briefly about the design choices that led me to openstack, and then about how to prepare for and install openstack. It seems like this guide is more like "you can use openstack for cloud". Needs more "guide".
- lioeters 4y ago
  According to the Diataxis model of documentation [0], the linked project is a "reference", not a "guide". It's a reference because it's information-oriented, and serves users who want to look up something based on category or keyword. A "guide for getting started" would be a tutorial, serving the needs of the user who is at study. It should provide a successful learning experience. The project would also benefit from how-to guides, to serve the needs of the user who have a certain task in mind that they want to accomplish. [0] https://diataxis.fr/ https://diataxis.fr/
netfortius 4y ago
Selfhosted subreddit [1] and its wiki [2] (which happens to be truer to a "getting started" concept, rather than the HN link), is all you'll ever need for the "self" approach. [1] https://old.reddit.com/r/selfhosted/ https://old.reddit.com/r/selfhosted/ [2] https://wiki.r-selfhosted.com/ https://wiki.r-selfhosted.com/
- johnklos 4y ago
  I don't think I'd recommend that subreddit to anyone who wants to get in to self hosting. They're quite gatekeepery unless you're doing things the way they want. Just look up any email self hosting there to see what I mean.
  - netfortius 4y ago
    Been there for a long while, and worked through self-hosting with a lot of cooperative efforts. What some may not like, or what I don't, recommended by others, was and is - in the end - upon my decision to implement. All in all - starting from their wiki is a far better choice than the original HN article.
TDiblik 4y ago
Hi, just asking around, I'm trying to self host on a basic Ubuntu server, I'll run some websites (behind nginx), databases (docker), cron jobs and other things on there. Normally, I don't really care about ddos, however since this is going to run on my home network (on it's own vlan separate from my vlan) and I have static IP, I'm kinda scared? I usually hosted my stuff on cloud with ddos protection included. What are some security precautions I can take to prevent this from happening?
- bsenftner 4y ago
  Not sure is this is the best option today, but in a previous self hosting situation I relied on CloudFlare's DDOS protection. We survived several attacks thanks to them. One attack was a 3 days long, and CloudFlare worked fine the entire time, our regular customers were lightly impacted but could still work.
  - capableweb 4y ago
    I guess it depends on where you draw the limit, but if you tunnel all your traffic via an external proxy to your "self-hosted" platform, how much can you really call that solution self-hosted? Did you try anything else before slapping cloudflare in front of your setup?
    - bsenftner 4y ago
      At the time, I was running a game service, so had a fair number of script kiddies attempting whatever. Expecting them, before using Cloudflare I got a hardware firewall. With a bit of research I located the same hardware firewall used by Federal Reserve Banks. At that time, an outright purchase of that thing was $24K, but it was well worth it. It had every bell and whistle one could ask: deep pack inspection, load balancing, and a quite a bit more. When I closed, that hardware firewall sold for the same price I paid, years earlier.
- wswope 4y ago
  Safest move would be to not actually expose your server from your public IP, but instead tunnel traffic from your home server through a cheap, public-facing VPS instance acting as a gateway host. To fully automate, you can use a premade tool like autossh, or just replicate it using standard ssh with keepalive option + a custom sysd unit file.
superkuh 4y ago
Containers and deployment and all that jazz. This is the exact kind of cargo cult nonsense that drives people away from self hosting due to the apparent complexity. NONE of that is needed to self host a website. Just install nginx from your repos and <html><head><title>My first website</title></head><body><H1>Wow! Hi.</H1></body></html> into a index.html and put in in your www directory. Done. And done in a far more secure, far longer lasting, far less fragile way than anything involving containers or "deployment".
- r1ch 4y ago
  Yup. I remember messing around with Redhat when I was 13. I installed Apache, edited the default index.html in htdocs then went to another computer, put in my IP and it worked! I was amazed, here I was running my own website. 25+ years later and the same steps are still possible to get the same result. I very much doubt k8s and Docker and whatever fancy new technology of the month is being pushed will have the same longevity. Simple technology rules.
  - PubliusMI 4y ago
    [flagged]
    - 533474 4y ago
      He is entitled to his opinion and you may be breaking the rules of HN by starting a flame war. Our technology is built upon the foundations from our predecessors. I am younger too but I do not promote ageism that is so rampant in our industry
      - horsawlarway 4y ago
        His opinion is misinformed and ignorant of the reality of hosting open source applications. If you want to host your own blog... sure, run a local copy of nginx or apache. That's not what this conversation is about. We're not hosting our own websites - we're replacing crappy ad driven/subscription SaaS companies with open source applications, running on our own hardware/networks. The use case there for containers is both obvious and compelling, and his comment shows that he has zero understanding of the space. So he's making an uninformed rant while talking about the "good ol' days" when everything was SO simple. Will Docker, specifically, be around in 20 years? Who gives a shit. Will the concept of an isolated & preconfigured runtime environment (containers)? Fuck yes it will.
boxed 4y ago
A better guide would be to just link to the dokku docs.
Test0129 4y ago
Silly. I've been self hosting since I degoogled. Never have I regretted not using docker. To imply that you should do this is the kind of hilariously out of touch I expect from developers :). This guide lacks the most important tools: fail2ban and tripwire. I block on the order of 5,000 malicious requests a day (mostly recon bots) from Russian and China.
- Beltalowda 4y ago
  I just run ssh on a different port. "Security by obscurity" works reasonably well to weed out the low-effort script kiddie nonsense.
  - pddpro 4y ago
    Or port-knocking.
- horsawlarway 4y ago
  To be fair - who gives a shit about 5k requests to mostly non-existent services? Even my dinky RPi can handle thousands of times more load, and the nginx proxy for the cluster is just going to hand back an empty 404. As long as you aren't doing silly shit (like exposing password based ssh, or running your containers as root, or failing to make backups) then they can knock all they'd like. They'd have to first find the app you're running, then attack the application to get an exploit, then manage to find a privilege escalation from the container, all for what, exactly? Send more requests from my network? Break a couple of my services that I don't derive revenue from? I'm just literally not worth the trouble in almost all cases. It's like 10 minutes to recreate my whole cluster from clean images (20 if we include wiping the OS on the machines). If you're really concerned - shove the cluster behind tailscale/cloudflare/vps.
  - Test0129 4y ago
    I really wasn't bragging about load. Of course 5k requests is nothing. It was mostly to emphasize that those 5k requests would bring at least 5k pieces of malware if I didn't spend the time to stop them. The guide was missing this which I think cheapens the guide a little to me because security can be a large portion of the cost of a SaaS service you might pay for.
intull 4y ago
I dislike how mean and critical HN can get sometimes. The article is clearly titled "A guide", not "The guide"! The OP link builds on top of using containerization / docker. Some find it useful, some would benefit from it. Doesn't mean that embrace is "silly", "nonsense", "cult", and whatnot. If you don't prefer something, make that known, that's fine. If there's something to add or alternatives to suggest, do so. One doesn't have to shitpile on a fairly well written, referenced and organized article.
psychphysic 4y ago
This is just one of those "awesome" lists. Recently I've been looking through them including the self hosted one and I suspect about 10-20% are in fact ads basically paid product placement. Usually can't even be self hosted just a paid service for an instance. That's not self hosting. A decent chunk are not practically selfhostable. I mean not for a mere mortal. Like those guys who seem to run synapse on a rpi3. Everyone else needs dual core with 6gb RAM. Again I wonder if those people actually even used the self hosted version? Am I still ranting? Guess I had more to say than I realised.

143 points by wagslane 4y ago

Maybe I missed it, but I don't see anything about discoverability. That's a big problem for servers on residential IP ranges.

abriosi 4y ago
This is one of the challenges I have faced with russing servers on local IPs. Do you have any resources you can share about this subject?
- woxerrat 4y ago
  https://freemyip.com/ https://freemyip.com/ is a good tool when on residential network with no static ip from ISP.
  - feet 4y ago
    Or you can rent a vps for $5 per month and wireguard your internal services to the vps. Vps gets put in the DNS records and throw traefik or caddy on the vps to handle things
    - squarefoot 4y ago
      Some HN user months ago suggested the entry level VPS offer from ionos.com which is $2/month at the moment. I bookmarked the link because I will probably need it when I relocate, but never used them so far. Any comments from customers are welcome. https://www.ionos.com/servers/vps#plans https://www.ionos.com/servers/vps#plans
jve 4y ago
I'd also like to learn more about about this subject. Cloudflare offers some API to update DNS: https://developers.cloudflare.com/dns/manage-dns-records/how-to/managing-dynamic-ip-addresses/ https://developers.cloudflare.com/dns/manage-dns-records/how... I'd like to see some of those scripts that use CF API to detect and update IP in dynamics public IP environment.
- thedanbob 4y ago
  It’s not particularly pretty, but here is my home-grown script for doing exactly that. It coordinates between Cloudflare DNS, my Unifi Security Gateway, and haproxy running on my home servers to keep my IPv4 and IPv6 addresses up to date everywhere. https://gist.github.com/thedanbob/df4cc49aa3e60581e010f443acfe254e https://gist.github.com/thedanbob/df4cc49aa3e60581e010f443ac...
- jcynix 4y ago
  If you have a server with a public address, a machine with a private address can simply use e.g. curl to access some (even non-existant) url on the server periodically which can then be extracted from the logs by a script on the server. I actually use this to announce my internal address, even without a web server, just by accessing a locked port on the server, where the firewall logs this access.
- watchdogtimer 4y ago
  I used to use Cloudflare's API to do this, but then we changed ISPs and now are behind CGNAT, so we don't even have a public IP. To publicly expose my web server, I created a Cloudflared tunnel, and have our DNS (which is also hosted at Cloudfare) point to the tunnel. It works quite well, and adds some security because we're no longer visible to attackers using IP address scans.
deniska 4y ago
I have a small script ran by cron (well, systemd timers) every 15 minutes which checks the current IP via https://api.ipify.org/ https://api.ipify.org/ and updates dns entries on digital ocean (its dns service is free) if it differs from what's there.
- KronisLV 4y ago
  If someone has to deal with dynamic IP addresses and wants to use dynamic DNS as well, there's also the pretty nice ddclient package: https://ddclient.net/ https://ddclient.net/ I've been using it with NameCheap pretty successfully for a while now, here's an example of some of the integrations it supports: https://ddclient.net/protocols.html https://ddclient.net/protocols.html (sadly no DigitalOcean integration as of yet, but this might be of use to others)
cassianoleal 4y ago
Self-hosting means hosting things yourself. If you don't know what you host that's a problem. Otherwise, discovery is largely irrelevant and residential IPs have no bearing on it. For actual reachability, a lot of people use VPNs, network meshes (Tailscale, ZeroTier, Nebula, etc) or secure proxies (Cloudflare Argo Tunnels). You can also self-host software on VPSs or cloud servers.
superkuh 4y ago
Discoverability? Uh... I think you may be dragging commercial concepts into this that don't have a place here. For a personal website people discover it when you link them to it. Or discover it 4 years later after google has indexed it and it happens organically. But you don't ever have to care about "discoverability". You're not trying to get a bunch of hits, or profit, or whatever. A personal website is something you do for fun. If this personal website is serving as your resume to get hired or sell stuff, it's not a personal website. It's more commercial/profit crap. At that point, yeah, use all the cargo cult you want. Business want to see the tools and technologies they use demonstrated. I've been hosting superkuh.com from my home PC for 20 years. When my IP changes (maybe once per year) I just log into my registrar when I notice it (it's okay if I don't notice for a day or two, no big deal) and update it.

dopidopHN 4y ago

No yunohost ?

readingnews 4y ago

Maybe I missed it, but this just looks like a giant list of open source tools. Docker, Containers, virtualization, password management, etc. Not knocking OP, but a "guide for getting started" seems like it would be a howto for self hosting. This seems like a giant list of tools. I only note this as I work at a place that self-hosts everything as we already have an on-prem data center. For example, we use openstack for our users to spin up VMs. If I did a guide to self hosting, I would talk briefly about the design choices that led me to openstack, and then about how to prepare for and install openstack. It seems like this guide is more like "you can use openstack for cloud". Needs more "guide".

lioeters 4y ago
According to the Diataxis model of documentation [0], the linked project is a "reference", not a "guide". It's a reference because it's information-oriented, and serves users who want to look up something based on category or keyword. A "guide for getting started" would be a tutorial, serving the needs of the user who is at study. It should provide a successful learning experience. The project would also benefit from how-to guides, to serve the needs of the user who have a certain task in mind that they want to accomplish. [0] https://diataxis.fr/ https://diataxis.fr/

netfortius 4y ago

Selfhosted subreddit [1] and its wiki [2] (which happens to be truer to a "getting started" concept, rather than the HN link), is all you'll ever need for the "self" approach. [1] https://old.reddit.com/r/selfhosted/ https://old.reddit.com/r/selfhosted/ [2] https://wiki.r-selfhosted.com/ https://wiki.r-selfhosted.com/

johnklos 4y ago
I don't think I'd recommend that subreddit to anyone who wants to get in to self hosting. They're quite gatekeepery unless you're doing things the way they want. Just look up any email self hosting there to see what I mean.
- netfortius 4y ago
  Been there for a long while, and worked through self-hosting with a lot of cooperative efforts. What some may not like, or what I don't, recommended by others, was and is - in the end - upon my decision to implement. All in all - starting from their wiki is a far better choice than the original HN article.

TDiblik 4y ago

Hi, just asking around, I'm trying to self host on a basic Ubuntu server, I'll run some websites (behind nginx), databases (docker), cron jobs and other things on there. Normally, I don't really care about ddos, however since this is going to run on my home network (on it's own vlan separate from my vlan) and I have static IP, I'm kinda scared? I usually hosted my stuff on cloud with ddos protection included. What are some security precautions I can take to prevent this from happening?

bsenftner 4y ago
Not sure is this is the best option today, but in a previous self hosting situation I relied on CloudFlare's DDOS protection. We survived several attacks thanks to them. One attack was a 3 days long, and CloudFlare worked fine the entire time, our regular customers were lightly impacted but could still work.
- capableweb 4y ago
  I guess it depends on where you draw the limit, but if you tunnel all your traffic via an external proxy to your "self-hosted" platform, how much can you really call that solution self-hosted? Did you try anything else before slapping cloudflare in front of your setup?
  - bsenftner 4y ago
    At the time, I was running a game service, so had a fair number of script kiddies attempting whatever. Expecting them, before using Cloudflare I got a hardware firewall. With a bit of research I located the same hardware firewall used by Federal Reserve Banks. At that time, an outright purchase of that thing was $24K, but it was well worth it. It had every bell and whistle one could ask: deep pack inspection, load balancing, and a quite a bit more. When I closed, that hardware firewall sold for the same price I paid, years earlier.
wswope 4y ago
Safest move would be to not actually expose your server from your public IP, but instead tunnel traffic from your home server through a cheap, public-facing VPS instance acting as a gateway host. To fully automate, you can use a premade tool like autossh, or just replicate it using standard ssh with keepalive option + a custom sysd unit file.

superkuh 4y ago

Containers and deployment and all that jazz. This is the exact kind of cargo cult nonsense that drives people away from self hosting due to the apparent complexity. NONE of that is needed to self host a website. Just install nginx from your repos and <html><head><title>My first website</title></head><body><H1>Wow! Hi.</H1></body></html> into a index.html and put in in your www directory. Done. And done in a far more secure, far longer lasting, far less fragile way than anything involving containers or "deployment".

r1ch 4y ago
Yup. I remember messing around with Redhat when I was 13. I installed Apache, edited the default index.html in htdocs then went to another computer, put in my IP and it worked! I was amazed, here I was running my own website. 25+ years later and the same steps are still possible to get the same result. I very much doubt k8s and Docker and whatever fancy new technology of the month is being pushed will have the same longevity. Simple technology rules.
- PubliusMI 4y ago
  [flagged]
  - 533474 4y ago
    He is entitled to his opinion and you may be breaking the rules of HN by starting a flame war. Our technology is built upon the foundations from our predecessors. I am younger too but I do not promote ageism that is so rampant in our industry
    - horsawlarway 4y ago
      His opinion is misinformed and ignorant of the reality of hosting open source applications. If you want to host your own blog... sure, run a local copy of nginx or apache. That's not what this conversation is about. We're not hosting our own websites - we're replacing crappy ad driven/subscription SaaS companies with open source applications, running on our own hardware/networks. The use case there for containers is both obvious and compelling, and his comment shows that he has zero understanding of the space. So he's making an uninformed rant while talking about the "good ol' days" when everything was SO simple. Will Docker, specifically, be around in 20 years? Who gives a shit. Will the concept of an isolated & preconfigured runtime environment (containers)? Fuck yes it will.

boxed 4y ago

A better guide would be to just link to the dokku docs.

Test0129 4y ago

Silly. I've been self hosting since I degoogled. Never have I regretted not using docker. To imply that you should do this is the kind of hilariously out of touch I expect from developers :). This guide lacks the most important tools: fail2ban and tripwire. I block on the order of 5,000 malicious requests a day (mostly recon bots) from Russian and China.

Beltalowda 4y ago
I just run ssh on a different port. "Security by obscurity" works reasonably well to weed out the low-effort script kiddie nonsense.
- pddpro 4y ago
  Or port-knocking.
horsawlarway 4y ago
To be fair - who gives a shit about 5k requests to mostly non-existent services? Even my dinky RPi can handle thousands of times more load, and the nginx proxy for the cluster is just going to hand back an empty 404. As long as you aren't doing silly shit (like exposing password based ssh, or running your containers as root, or failing to make backups) then they can knock all they'd like. They'd have to first find the app you're running, then attack the application to get an exploit, then manage to find a privilege escalation from the container, all for what, exactly? Send more requests from my network? Break a couple of my services that I don't derive revenue from? I'm just literally not worth the trouble in almost all cases. It's like 10 minutes to recreate my whole cluster from clean images (20 if we include wiping the OS on the machines). If you're really concerned - shove the cluster behind tailscale/cloudflare/vps.
- Test0129 4y ago
  I really wasn't bragging about load. Of course 5k requests is nothing. It was mostly to emphasize that those 5k requests would bring at least 5k pieces of malware if I didn't spend the time to stop them. The guide was missing this which I think cheapens the guide a little to me because security can be a large portion of the cost of a SaaS service you might pay for.

intull 4y ago

I dislike how mean and critical HN can get sometimes. The article is clearly titled "A guide", not "The guide"! The OP link builds on top of using containerization / docker. Some find it useful, some would benefit from it. Doesn't mean that embrace is "silly", "nonsense", "cult", and whatnot. If you don't prefer something, make that known, that's fine. If there's something to add or alternatives to suggest, do so. One doesn't have to shitpile on a fairly well written, referenced and organized article.

psychphysic 4y ago

This is just one of those "awesome" lists. Recently I've been looking through them including the self hosted one and I suspect about 10-20% are in fact ads basically paid product placement. Usually can't even be self hosted just a paid service for an instance. That's not self hosting. A decent chunk are not practically selfhostable. I mean not for a mere mortal. Like those guys who seem to run synapse on a rpi3. Everyone else needs dual core with 6gb RAM. Again I wonder if those people actually even used the self hosted version? Am I still ranting? Guess I had more to say than I realised.