11 ms·

I self-host literally everything (email, calendar/contacts, VOIP, XMPP, you name it) from by basement with used 1U servers from eBay and a cable internet connec

I self-host literally everything (email, calendar/contacts, VOIP, XMPP, you name it) from by basement with used 1U servers from eBay and a cable internet connection. It was probably more hassle than most people would want to bother with to get it set up. But, with everything up and running, there's very little maintenance. I probably spend a few hours a month tinkering still, just because I enjoy it. I use a stack of Proxmox VMs, FreeIPA for authn/authz, and Rocky Linux for all servers and workstations. My phone runs GrapheneOS with a Wireguard VPN back to the house. I don't expose anything to the public internet unless absolutely necessary. I recently anonymized and Ansibilized my entire setup so that others might get some use out of it: https://github.com/sacredheartsc/selfhosted https://github.com/sacredheartsc/selfhosted

triyambakam 4y ago
Very inspiring and thank you for sharing. I run GrapheneOS too but I haven't set anything up like a Wireguard VPN. What is the rough idea of how that works?
- stonewall 4y ago
  I plug my cable modem into a server running the OPNsense firewall [0], which has a wireguard plugin. I set up a wireguard VPN in OPNsense. Then I downloaded the wireguard app in F-Droid, and pasted my credentials from the wireguard Android app into the wireguard configs on the firewall. I set the VPN in grapheneOS as "always on," so from my phone's perspective, it always has access to my internal network, even when on LTE. All my phones internet traffic ends up going through my home internet connection as a result. [0] https://opnsense.org/ https://opnsense.org/
- j45 4y ago
  Try installing algovpn it’s pretty much a turnkey wireguard installation, lots of tutorials on YouTube. I would advise against setting up wireguard manually.
- zwilliamson 4y ago
  Checkout Tailscale for an easy to rollout WireGuard based solution that has a fair free tier
novok 4y ago
How much power does it take? I've realized with some services it's cheaper to use it than the electricity and hardware cost.
- stonewall 4y ago
  I almost certainly don't save any money considering electricity cost. I have a dell r630 for compute and an r730xd that I use as a NAS. Then I have one switch for the rack and a POE switch for the house. Probably 3-5amps total? If I started over, I would probably choose more efficient gear. That said, I don't mind paying for the electricity too much. I enjoy the warm fuzzies of knowing my data lives under my roof.
  - vineyardmike 4y ago
    > Probably 3-5amps total? A raspberry pi draws 2+ amps. Your dual Xeon server is drawing a lot more power. That said, typically you’d want to measure in watts because amps is relative. Eg a RPI is 2A at 5V while a computer is probably 5A at 120V - an order of magnitude more total energy consumed.
  - chinaman425 4y ago
    [dead]
  - pmarreck 4y ago
    do you backup offsite? if not, in the event of a fire, your data will live under your "poof!"
    - stonewall 4y ago
      I have some automation that does a weekly archive of everything important to a ZFS-based NAS. Home directories are also stored there over NFS, with hourly/weekly/monthly snapshots. Once a month or so, I plug in two separate 5TB external HDDs and run a backup script that rsync's everything to each one (2 is 1 and 1 is none). These are stored outside my home. I should probably get some kind of cloud-based / encrypted backup thing going as well. I don't claim that my current backup system is very good.
ryjo 4y ago
Incredible. The usual response to "should I host my own email" is "don't do it; you'll get hacked." Three questions: 1. Have you heard of this complaint? 2. Do you use a home ISP connection, or a commercial ISP connection? A "home ISP connection" here usually comes with a dynamic IP address; you can't get your hands on a static address without paying a very large amount monthly or getting a commercial connection. 3. You say "I don't expose anything to the public internet unless absolutely necessary." Is your ip address via your domain name one of those "necessary" items?
- stonewall 4y ago
  1. Yes, most people will tell you not to host your own email, because its too complicated/difficult to get your mail delivered reliably. A lot of this is FUD. Yes, email is a bit more difficult to get right than say, hosting a web app behind Nginx. It's an old protocol, with many "features" bolted on years later to combat spam. I'm not sure how email is easier to "hack," unless there is a zero day in Postfix or something. Back in the day, lots of script kiddies would find poorly configured mail servers that were happy to act as an open relay...maybe the stigma persists? To deliver mail reliably, you need 4 things (in my experience): - A static, public IP address with a good reputation (ie, not on any spam blacklists) - A reverse DNS record that resolves back to your mail server's IP - A domain SPF record that says that your mail server is allowed to deliver mail - DKIM records and proper signing of outgoing messages (DMARC records help too) 2. I have a residential cable internet connection, but pay extra for static IPs. You can probably get by with a dynamic IP and some kind of dynamic DNS service, as long as you don't want to send email. You could still receive email locally if your MX recorded pointed to some kind of dynamic DNS record. Note that some ISPs explicitly block outbound traffic on port 25 due to spammers. You might need to check with yours. 3. The only things I expose to the internet are Postfix (to send/receive emails), XMPP (to chat with others), and my web server. Everything else (calendar/contacts, IMAP, Syncthing, etc) stays behind my firewall, accessible only to internal hosts. I use wireguard on my Android phone to access these services seamlessly when I leave the house. I've never bothered to conceal my IP address. For awhile, I experimented with using Mullvad VPN for all my egress traffic. Unfortunately I spent all day solving CAPTCHAs...wasn't worth it (for me, anyway). EDIT: I should add, that I also have a "normie" email address at one of the usual providers that I use for really important things like bank accounts / utility providers. If I get hit by a bus, I don't want my (very nontechnical) wife to deal with sysadminning on top of my early death. For all our personal communications though, we use my selfhosted email domain.
xyzzy123 4y ago
I had fun doing this until I had kids. I have a rack with 10gbe, ups, kubernetes a zfs storage server, multiple vlans, 4 unifi APs & locally hosted controller and all sorts of self-hosted stuff. My heart breaks slightly as I watch things slowly degrade and break down due to bit-rot and version creep, I now wish I had a synology, flat network and cloud everything possible. There are days when the kids can't watch a particular movie and I find out it's because a particular kube component failed (after an hour of root-causing) because I haven't touched it in 2 years. I then have regrets about my life choices. Sometimes the rack starts beeping while I'm working and I realise the UPS batteries are due for replacement because it's been 4 years. I silence the alarm and get back to the production issue at work, knowing it'll beep at me again in 30 days. I'll still be too busy to fix it. It doesn't help that in Australia the ambient can get to 45 degrees C pushing disks and cpus to their limits. Just sharing a different perspective...
- logifail 4y ago
  > I watch things slowly degrade and break down due to bit-rot and version creep [..] > There are days when the kids can't watch a particular movie and I find out it's because a particular kube component failed (after an hour of root-causing) because I haven't touched it in 2 years. I then have regrets about my life choices. [..] > I now wish I had a synology, flat network and cloud everything possible No snark intended, but this sounds as though you chose to include a lot of unnecessary complexity into your self-hosting, then discovered that there's almost always a cost to unnecessary complexity(?)
- stonewall 4y ago
  Your perspective resonates with me! I have 3 kids under 6 years old, and I can definitely see this easily creeping up in my future. My family situation is partly why I just went with plain old VMs and a Linux Distro with a 10 year support cycle. Its easy to keep all the moving parts in my head, and I figure I can mostly coast for 10 years and then reevaluate. Thanks for reminding me, I also need to replace my UPS battery...
- tharkun__ 4y ago
  Sounds like a bit of overkill too if you ask me. You can self-host most things that make sense to keep private without going all in on the fun stuff. As in, k8s is cool to play with and understand and all but why would I bring that complexity to a simple home setup that can run on a single machine in a corner somewhere? You don't have to go to a synology box and give up everything but there are simpler options without going "Cloud everything". Of course you will be giving up some features as well, the more you strip things down, but that can beneficial in and of itself if you ask me. Personally I went from being the "Linux from scratch" guy to running Ubuntu LTS. Natural progression and the kids can watch any of their movies at any time they want. Keep the hard drives rotated, do an LTS to LTS upgrade every few years and that's about it. Heck I've been running the exact same Postfix, fetchmail and IMAP setup for probably 20 years now and I don't even remember what all the options I set do any longer. I also don't need to though. It's just rock solid. All the other fun stuff has passed me by and I don't care. Don't get me wrong, it's still fun to play with stuff and we do use k8s at work and it's great. But it's just complete overkill for home.
hinkley 4y ago
We need to work on a mostly turnkey solution for these things. I still think another generation or two of raspi and friends and you can build a little cluster of them.
beaukin 4y ago
This GitHub share is pure gold. You’re amazing.
- scruple 4y ago
  Agreed. It's divine compared to the janky Ansible setups I've seen in the wild.
- stonewall 4y ago
  Thank you for the kind words :)
zamnos 4y ago
What do you do for backups? If your house gets destroyed in a natural disaster, will all your pictures persist?
- stonewall 4y ago
  I regularly back up to some external HDDs that I keep outside the home. For pictures specifically, I recently discovered M-Disc [0], which are (allegedly) archival-quality, writable Blu-Ray discs. I'm considering burning an M-Disc of each year's pictures and storing them in jewel cases at a family member's house. [0] https://www.mdisc.com/ https://www.mdisc.com/
  - zamnos 4y ago
    > some external HDDs that I keep outside the home. Personally, I'm not remotely meticulous enough for that to work. Properly rotating drives sounds like a lot of work if you want to be rigorous about it. You start with running the backup to drive A, then shipping that drive a couple hundred miles away (to be properly location redundant), and then next week, run the backup to drive B, ship that drive a couple hundred miles away, but then at some point, you're going to want drive A back, so you can rotate drives and put a more recent backups on it. How do you retrieve those external drives, and consistently? And then while the drive is in transit, and hopefully not lost, you don't have access to it, it's not an online (referring to its availability) backup solution. So I mean, I do perform backups to an external HDs which I also keep offsite, but because that's nowhere near as rigorous as what teams of engineers and data center techs can do with a much larger budget, I supplement my backups with a cloud storage solution. And I encourage you to do so as well (especially considering encrypted backup services), but you do you. As far as the mdisc; I mean it's interesting, but I'd also consider getting an LTO tape library. They're more purpose build for backing things up, and my personal opinion is they're going to be better for longevity given everything else.

triyambakam 4y ago

Very inspiring and thank you for sharing. I run GrapheneOS too but I haven't set anything up like a Wireguard VPN. What is the rough idea of how that works?

stonewall 4y ago
I plug my cable modem into a server running the OPNsense firewall [0], which has a wireguard plugin. I set up a wireguard VPN in OPNsense. Then I downloaded the wireguard app in F-Droid, and pasted my credentials from the wireguard Android app into the wireguard configs on the firewall. I set the VPN in grapheneOS as "always on," so from my phone's perspective, it always has access to my internal network, even when on LTE. All my phones internet traffic ends up going through my home internet connection as a result. [0] https://opnsense.org/ https://opnsense.org/
j45 4y ago
Try installing algovpn it’s pretty much a turnkey wireguard installation, lots of tutorials on YouTube. I would advise against setting up wireguard manually.
zwilliamson 4y ago
Checkout Tailscale for an easy to rollout WireGuard based solution that has a fair free tier

novok 4y ago

How much power does it take? I've realized with some services it's cheaper to use it than the electricity and hardware cost.

stonewall 4y ago
I almost certainly don't save any money considering electricity cost. I have a dell r630 for compute and an r730xd that I use as a NAS. Then I have one switch for the rack and a POE switch for the house. Probably 3-5amps total? If I started over, I would probably choose more efficient gear. That said, I don't mind paying for the electricity too much. I enjoy the warm fuzzies of knowing my data lives under my roof.
- vineyardmike 4y ago
  > Probably 3-5amps total? A raspberry pi draws 2+ amps. Your dual Xeon server is drawing a lot more power. That said, typically you’d want to measure in watts because amps is relative. Eg a RPI is 2A at 5V while a computer is probably 5A at 120V - an order of magnitude more total energy consumed.
- chinaman425 4y ago
  [dead]
- pmarreck 4y ago
  do you backup offsite? if not, in the event of a fire, your data will live under your "poof!"
  - stonewall 4y ago
    I have some automation that does a weekly archive of everything important to a ZFS-based NAS. Home directories are also stored there over NFS, with hourly/weekly/monthly snapshots. Once a month or so, I plug in two separate 5TB external HDDs and run a backup script that rsync's everything to each one (2 is 1 and 1 is none). These are stored outside my home. I should probably get some kind of cloud-based / encrypted backup thing going as well. I don't claim that my current backup system is very good.

ryjo 4y ago

Incredible. The usual response to "should I host my own email" is "don't do it; you'll get hacked." Three questions: 1. Have you heard of this complaint? 2. Do you use a home ISP connection, or a commercial ISP connection? A "home ISP connection" here usually comes with a dynamic IP address; you can't get your hands on a static address without paying a very large amount monthly or getting a commercial connection. 3. You say "I don't expose anything to the public internet unless absolutely necessary." Is your ip address via your domain name one of those "necessary" items?

stonewall 4y ago
1. Yes, most people will tell you not to host your own email, because its too complicated/difficult to get your mail delivered reliably. A lot of this is FUD. Yes, email is a bit more difficult to get right than say, hosting a web app behind Nginx. It's an old protocol, with many "features" bolted on years later to combat spam. I'm not sure how email is easier to "hack," unless there is a zero day in Postfix or something. Back in the day, lots of script kiddies would find poorly configured mail servers that were happy to act as an open relay...maybe the stigma persists? To deliver mail reliably, you need 4 things (in my experience): - A static, public IP address with a good reputation (ie, not on any spam blacklists) - A reverse DNS record that resolves back to your mail server's IP - A domain SPF record that says that your mail server is allowed to deliver mail - DKIM records and proper signing of outgoing messages (DMARC records help too) 2. I have a residential cable internet connection, but pay extra for static IPs. You can probably get by with a dynamic IP and some kind of dynamic DNS service, as long as you don't want to send email. You could still receive email locally if your MX recorded pointed to some kind of dynamic DNS record. Note that some ISPs explicitly block outbound traffic on port 25 due to spammers. You might need to check with yours. 3. The only things I expose to the internet are Postfix (to send/receive emails), XMPP (to chat with others), and my web server. Everything else (calendar/contacts, IMAP, Syncthing, etc) stays behind my firewall, accessible only to internal hosts. I use wireguard on my Android phone to access these services seamlessly when I leave the house. I've never bothered to conceal my IP address. For awhile, I experimented with using Mullvad VPN for all my egress traffic. Unfortunately I spent all day solving CAPTCHAs...wasn't worth it (for me, anyway). EDIT: I should add, that I also have a "normie" email address at one of the usual providers that I use for really important things like bank accounts / utility providers. If I get hit by a bus, I don't want my (very nontechnical) wife to deal with sysadminning on top of my early death. For all our personal communications though, we use my selfhosted email domain.

xyzzy123 4y ago

I had fun doing this until I had kids. I have a rack with 10gbe, ups, kubernetes a zfs storage server, multiple vlans, 4 unifi APs & locally hosted controller and all sorts of self-hosted stuff. My heart breaks slightly as I watch things slowly degrade and break down due to bit-rot and version creep, I now wish I had a synology, flat network and cloud everything possible. There are days when the kids can't watch a particular movie and I find out it's because a particular kube component failed (after an hour of root-causing) because I haven't touched it in 2 years. I then have regrets about my life choices. Sometimes the rack starts beeping while I'm working and I realise the UPS batteries are due for replacement because it's been 4 years. I silence the alarm and get back to the production issue at work, knowing it'll beep at me again in 30 days. I'll still be too busy to fix it. It doesn't help that in Australia the ambient can get to 45 degrees C pushing disks and cpus to their limits. Just sharing a different perspective...

logifail 4y ago
> I watch things slowly degrade and break down due to bit-rot and version creep [..] > There are days when the kids can't watch a particular movie and I find out it's because a particular kube component failed (after an hour of root-causing) because I haven't touched it in 2 years. I then have regrets about my life choices. [..] > I now wish I had a synology, flat network and cloud everything possible No snark intended, but this sounds as though you chose to include a lot of unnecessary complexity into your self-hosting, then discovered that there's almost always a cost to unnecessary complexity(?)
stonewall 4y ago
Your perspective resonates with me! I have 3 kids under 6 years old, and I can definitely see this easily creeping up in my future. My family situation is partly why I just went with plain old VMs and a Linux Distro with a 10 year support cycle. Its easy to keep all the moving parts in my head, and I figure I can mostly coast for 10 years and then reevaluate. Thanks for reminding me, I also need to replace my UPS battery...
tharkun__ 4y ago
Sounds like a bit of overkill too if you ask me. You can self-host most things that make sense to keep private without going all in on the fun stuff. As in, k8s is cool to play with and understand and all but why would I bring that complexity to a simple home setup that can run on a single machine in a corner somewhere? You don't have to go to a synology box and give up everything but there are simpler options without going "Cloud everything". Of course you will be giving up some features as well, the more you strip things down, but that can beneficial in and of itself if you ask me. Personally I went from being the "Linux from scratch" guy to running Ubuntu LTS. Natural progression and the kids can watch any of their movies at any time they want. Keep the hard drives rotated, do an LTS to LTS upgrade every few years and that's about it. Heck I've been running the exact same Postfix, fetchmail and IMAP setup for probably 20 years now and I don't even remember what all the options I set do any longer. I also don't need to though. It's just rock solid. All the other fun stuff has passed me by and I don't care. Don't get me wrong, it's still fun to play with stuff and we do use k8s at work and it's great. But it's just complete overkill for home.

hinkley 4y ago

We need to work on a mostly turnkey solution for these things. I still think another generation or two of raspi and friends and you can build a little cluster of them.

beaukin 4y ago

This GitHub share is pure gold. You’re amazing.

scruple 4y ago
Agreed. It's divine compared to the janky Ansible setups I've seen in the wild.
stonewall 4y ago
Thank you for the kind words :)

zamnos 4y ago

What do you do for backups? If your house gets destroyed in a natural disaster, will all your pictures persist?

stonewall 4y ago
I regularly back up to some external HDDs that I keep outside the home. For pictures specifically, I recently discovered M-Disc [0], which are (allegedly) archival-quality, writable Blu-Ray discs. I'm considering burning an M-Disc of each year's pictures and storing them in jewel cases at a family member's house. [0] https://www.mdisc.com/ https://www.mdisc.com/
- zamnos 4y ago
  > some external HDDs that I keep outside the home. Personally, I'm not remotely meticulous enough for that to work. Properly rotating drives sounds like a lot of work if you want to be rigorous about it. You start with running the backup to drive A, then shipping that drive a couple hundred miles away (to be properly location redundant), and then next week, run the backup to drive B, ship that drive a couple hundred miles away, but then at some point, you're going to want drive A back, so you can rotate drives and put a more recent backups on it. How do you retrieve those external drives, and consistently? And then while the drive is in transit, and hopefully not lost, you don't have access to it, it's not an online (referring to its availability) backup solution. So I mean, I do perform backups to an external HDs which I also keep offsite, but because that's nowhere near as rigorous as what teams of engineers and data center techs can do with a much larger budget, I supplement my backups with a cloud storage solution. And I encourage you to do so as well (especially considering encrypted backup services), but you do you. As far as the mdisc; I mean it's interesting, but I'd also consider getting an LTO tape library. They're more purpose build for backing things up, and my personal opinion is they're going to be better for longevity given everything else.