8 ms·

I self-host the following at home. Everything is running under LXD (and I have all of the scripts to set it up here[1]): * nginx to reverse-proxy each of the

by cyphar 7y ago

I self-host the following at home. Everything is running under LXD (and I have all of the scripts to set it up here[1]): * nginx to reverse-proxy each of the services. * NextCloud. * Matrix Homeserver (synapse). * My website (dumb Flask webapp). * Tor (non-exit) relay. * Tor onion service for my website. * Wireguard VPN (not running in a container, obviously). All running on an openSUSE Leap box, with ZFS as the filesystem for my drives (simple stripe over 2-way mirrors of 4TB drives). It also acts as an NFS server for my media center (Kodi -- though I really am not a huge fan of LibreELEC) to pull videos, music, and audiobooks from. Backups are done using restic (and ZFS snapshots to ensure they're atomic) and are pushed to BackBlaze B2. I used to run an IRC bouncer but Matrix fills that need these days. I might end up running my own Gitea (or gitweb) server one day though -- I don't really like that I host everything on GitHub. I have considered hosting my own email server, but since this is all done from a home ISP connection that probably isn't such a brilliant idea. I just use Mailbox.org. [1]: https://github.com/cyphar/cyphar.com/tree/master/srv https://github.com/cyphar/cyphar.com/tree/master/srv

mwcampbell 7y ago
> It also acts as an NFS server for my media center [...] to pull videos, music, and audiobooks from. This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi? Do you also have a way of syncing them to a more portable device, like your phone?
- cyphar 7y ago
  > This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi? Sometimes, though not very often -- I work from home and so sometimes I'll play an audiobook in my living room and work at the dinner table rather than working from my home office. > Do you also have a way of syncing them to a more portable device, like your phone? Unfortunately not in an automated way (luckily I don't buy audiobooks very regularly -- I like to finish one before I get another one). I really wish that VLC on Android supported NFS, but it doesn't AFAIK (I think it requires kernel support).
mwcampbell 7y ago
I'm curious about why you're using lxd. Is it just that you wanted to try something different from Docker and its rivals? Or is there a reason you think lxd is better for your setup? For a service per container, I figured minimal, immutable containers, rather than containers running full distros, would be better.
- cyphar 7y ago
  The primary reason is that LXD has an indisputably better overall security policy than Docker. They support isolated user namespaces (containers running with different userns mappings), user namespaces are the default, they make use of far more new kernel hardening features than Docker, and so on. If I'm going to self-host something at home and expose it to the internet, I'm simply not going to use Docker. I used to run Docker containers several years ago, but I found them far more frustrating to manage. --restart policies were fairly hairy to make sure they actually worked properly, the whole "link" system in Docker is pretty frustrating to use, docker-compose has a laundry-list of problems, and so on. With LXD I have a fairly resilient setup that just requires a few proxy devices to link services together, and boot.autostart always works. Personally, I also find it much simpler to manage a couple of services as full-distro containers. Having to maintain your own Dockerfiles to work around bugs (and missteps) in the "official library" Docker images also added a bunch of senseless headaches. I just have a few scripts that will auto-set up a new LXD container using my configuration -- so I can throw away and recreate any one of my LXD containers. [Note: I do actually maintain runc -- which is the runtime underneath Docker -- and I've contributed to Docker a fair bit in the past. So all of the above is a bit more than just uneducated conjecture.]
  - gnfurlong 7y ago
    Have you taken a look at podman / buildah? My understanding is that podman resolves all of the security concerns you highlight above while mostly maintaining compatability with the docker cli and existing docker images. It gets rid of the docker daemon so your containers (and restart policy) can just be managed by your existing service manager. I only just recently discovered podman and I've been pretty excited. Having never used LXD and only understanding the high level differences between the two, I'm curious how it compares with regards to security and usability.
big_chungus 7y ago
Why SUSE over another OS? I've used it and like it, though I see more ubuntu, debian, centos among servers. Any particular distinguishing factor/advantage, or just preference?
- cyphar 7y ago
  I've worked for SUSE for quite a few years now, and so I've gotten fairly used to running openSUSE on all my machines (and I do quite like things like the Open Build Service and other openSUSE projects). I'm am a package maintainer for a bunch of openSUSE packages (most of the container-related ones and a few others) -- so I might as well use them myself to make sure they work properly.
douglascoding 7y ago
> * Wireguard VPN (not running in a container, obviously). I plan to use Wireguard too, so I shouldn't run on containers? Can you elaborate on that?
- BrandoElFollito 7y ago
  From the small research I did, I think you need a customized kernel on the host to do that. I run it on the host.

by cyphar 7y ago

mwcampbell 7y ago

> It also acts as an NFS server for my media center [...] to pull videos, music, and audiobooks from. This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi? Do you also have a way of syncing them to a more portable device, like your phone?

cyphar 7y ago
> This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi? Sometimes, though not very often -- I work from home and so sometimes I'll play an audiobook in my living room and work at the dinner table rather than working from my home office. > Do you also have a way of syncing them to a more portable device, like your phone? Unfortunately not in an automated way (luckily I don't buy audiobooks very regularly -- I like to finish one before I get another one). I really wish that VLC on Android supported NFS, but it doesn't AFAIK (I think it requires kernel support).

mwcampbell 7y ago

I'm curious about why you're using lxd. Is it just that you wanted to try something different from Docker and its rivals? Or is there a reason you think lxd is better for your setup? For a service per container, I figured minimal, immutable containers, rather than containers running full distros, would be better.

cyphar 7y ago
The primary reason is that LXD has an indisputably better overall security policy than Docker. They support isolated user namespaces (containers running with different userns mappings), user namespaces are the default, they make use of far more new kernel hardening features than Docker, and so on. If I'm going to self-host something at home and expose it to the internet, I'm simply not going to use Docker. I used to run Docker containers several years ago, but I found them far more frustrating to manage. --restart policies were fairly hairy to make sure they actually worked properly, the whole "link" system in Docker is pretty frustrating to use, docker-compose has a laundry-list of problems, and so on. With LXD I have a fairly resilient setup that just requires a few proxy devices to link services together, and boot.autostart always works. Personally, I also find it much simpler to manage a couple of services as full-distro containers. Having to maintain your own Dockerfiles to work around bugs (and missteps) in the "official library" Docker images also added a bunch of senseless headaches. I just have a few scripts that will auto-set up a new LXD container using my configuration -- so I can throw away and recreate any one of my LXD containers. [Note: I do actually maintain runc -- which is the runtime underneath Docker -- and I've contributed to Docker a fair bit in the past. So all of the above is a bit more than just uneducated conjecture.]
- gnfurlong 7y ago
  Have you taken a look at podman / buildah? My understanding is that podman resolves all of the security concerns you highlight above while mostly maintaining compatability with the docker cli and existing docker images. It gets rid of the docker daemon so your containers (and restart policy) can just be managed by your existing service manager. I only just recently discovered podman and I've been pretty excited. Having never used LXD and only understanding the high level differences between the two, I'm curious how it compares with regards to security and usability.

big_chungus 7y ago

Why SUSE over another OS? I've used it and like it, though I see more ubuntu, debian, centos among servers. Any particular distinguishing factor/advantage, or just preference?

cyphar 7y ago
I've worked for SUSE for quite a few years now, and so I've gotten fairly used to running openSUSE on all my machines (and I do quite like things like the Open Build Service and other openSUSE projects). I'm am a package maintainer for a bunch of openSUSE packages (most of the container-related ones and a few others) -- so I might as well use them myself to make sure they work properly.

douglascoding 7y ago

> * Wireguard VPN (not running in a container, obviously). I plan to use Wireguard too, so I shouldn't run on containers? Can you elaborate on that?

BrandoElFollito 7y ago
From the small research I did, I think you need a customized kernel on the host to do that. I run it on the host.