7 ms·

This is a pretty cool setup! I have been tinkering in the self-hosting world for awhile now and I would say my biggest piece of advice is to learn Docker. (Not

This is a pretty cool setup! I have been tinkering in the self-hosting world for awhile now and I would say my biggest piece of advice is to learn Docker. (Not trying to shill for Docker here or start a Docker vs. Ansible flame war.) In my personal experience, Docker images (and docker-compose orchestrations) are more pervasive than Ansible playbooks and when it comes to self-hosting for personal use, hosting all my services in Docker has made it much easier to deploy/maintain them. (Not speaking to business use cases since in that situation you should really have a deep familiarity with your tech stack and be able to roll-your-own Ansible playbook/Dockerfiles.) Also, there are so many great FLOSS alternatives to Google Apps. This repo contains some, but here are some of my favorites: * Drive/Calendar/Photos/Keep: - https://nextcloud.com/ (I prefer this over OwnCloud) * Docs/Sheets/Slides - https://www.onlyoffice.com/ * Mail - https://mailu.io/ (basically a Docker-based deployment of Postfix/Dovecot/etc) * Hangouts - Server - https://matrix.org/ - Client - https://about.riot.im/ - (I prefer Matrix.org over Jabber/XMPP) * G+/Twitter - https://joinmastodon.org/

tvanantwerp 7y ago
Are there any resources you recommend for learning to use Docker specifically for self-hosted services in a home lab setting?
- jlkuester7 7y ago
  IMHO the official Docker documentation is great! Their "getting started" page is a good overview of the Docker basics. https://docs.docker.com/get-started/ https://docs.docker.com/get-started/ The CLI --help is pretty useful too. I have also found Docker's in-depth documentation to be very helpful when trying to figure out the more complex features. I got started by just grabbing a $5 DigitalOcean droplet (can get them with Docker pre-installed) and then played around trying to setup a simple app. (I think it was RocketChat.) https://rocket.chat/docs/installation/docker-containers/ https://rocket.chat/docs/installation/docker-containers/
- orionhasyou 7y ago
  https://www.linuxserver.io/ https://www.linuxserver.io/ not a resource but docker containers you might want to run in your homelab. checkout awesome docker on github or katacoda
lxcfan 7y ago
The two are not incompatible. You can deploy containers with Ansible.
- yjftsjthsd-h 7y ago
  Indeed:) Ansible will happily interface with pretty much anything; consider that it officially supports Swarm (https://docs.ansible.com/ansible/latest/modules/docker_swarm_service_module.html https://docs.ansible.com/ansible/latest/modules/docker_swarm...), k8s (https://docs.ansible.com/ansible/latest/modules/k8s_module.html https://docs.ansible.com/ansible/latest/modules/k8s_module.h...), and puppet (https://docs.ansible.com/ansible/latest/modules/puppet_module.html https://docs.ansible.com/ansible/latest/modules/puppet_modul...) for example:) It makes excellent glue if used with other things, or a good tool all on its own. Which is especially nice because it lets you try new things without changing your overall setup too much; you can deploy all your services with RPMs, and then move one thing into docker, and still use ansible for everything:)
ggregoire 7y ago
> Not trying to start a Docker vs. Ansible flame war Don't worry, they serve very different purposes. You already probably know but Docker is for running applications in isolation, while Ansible is for provisioning and configuring hosts. For instance, you won't use Docker to harden sshd on your hosts but Ansible.
- bwbmr 7y ago
  Are there any good resources (besides OP) for using Ansible in a home-server environment? Especially in a mixed docker + Ansible environment.
  - jefurii 7y ago
    After figuring out Ansible work I started using it to manage several personal VPS machines and the various boxes at home. I just used the Ansible docs, which are pretty good.
  - S04dKHzrKT 7y ago
    https://github.com/davestephens/ansible-nas https://github.com/davestephens/ansible-nas Uses ansible + docker to setup and run a home server. Its feature-set is close to that of a typical FreeNAS setup in a home environment.
  - ggregoire 7y ago
    The official doc, StackOverflow and /r/ansible are good resources imo.
  - VvR-Ox 7y ago
    Just start a simple project with Ansible and have a look at the repositories using Ansible to install Docker and deploy containers on your host afterwards. You may want to put some effort (not that much) into managing your credentials with Ansible Vault[0] and you can try your playbooks e.g. on a Vagrant Machine[1] before applying them to a real host. Any tutorial will do for the beginning but you should always notice what version of Ansible you are using (vs. the one used in the tutorial) as features change and also there have been some changes to the syntax to improve readability of your playbooks. [0]: https://docs.ansible.com/ansible/latest/user_guide/vault.html https://docs.ansible.com/ansible/latest/user_guide/vault.htm... [1]: https://www.vagrantup.com/docs/provisioning/ansible.html https://www.vagrantup.com/docs/provisioning/ansible.html
newscracker 7y ago
> I would say my biggest piece of advice is to learn Docker. (Not trying to shill for Docker here The first step to learning Docker by downloading it, irritatingly, is that getting Docker Desktop on Mac or Windows requires creating a Docker Hub account and signing in! There’s even a long issue thread about this on GitHub and the response was totally irrelevant! Luckily for those who don’t want to jump through these hoops (through disposable addresses or reusable shared logins) or provide an email address, there are many people who have posted direct links to the different downloads available.
- CraigJPerry 7y ago
  I think you can use chocolatey to choco install it without going to the docker site.
- DarthGhandi 7y ago
  I remember when they changed that and the GitHub issue was one of the most hilarious examples of corporate doublespeak. "Why are you doing this?" "To improve user experience" "But now I have to sign up for an account whereas before I could just download" "Yes this will improve the experience for everyone over time"
zzzeek 7y ago
I run everything in containers and I spin up all the images / containers from ansible scripts. Am I doing it wrong? I didn't think these two tools were in conflict.
- dcardoza 7y ago
  They do complement one another but docker has tooling that solves similar problems. Does docker-compose (https://docs.docker.com/compose/ https://docs.docker.com/compose/) fit your needs?
  - onei 7y ago
    I tend to run ansible to setup a VM, e.g. a few packages, lock it down to my taste, and then have ansible run docker-compose to deploy the application(s) of interest. You wouldn't use docker to manage the host networking (which is where ansible comes in), but packaging whatever is listening on ports as a container works really well for me
  - zzzeek 7y ago
    I use ansible to build a machine from zero software installed on it to running multiple docker containers that also make use of various filesystem stuff on the host, as well as set up other non-docker things on the host like networking, selinux stuff, and crons. I like that ansible is not hardwired towards everything being a container.
davestephens 7y ago
No need for an Ansible vs. Docker argument when you can combine them both! I run a similar project called Ansible-NAS - https://github.com/davestephens/ansible-nas https://github.com/davestephens/ansible-nas - which originally came about because I fell out of love with FreeNAS, and felt I could do a better job with Ubuntu, Ansible, and a bunch of Docker images. Sovereign is awesome, I've been watching it for a while, but I'm not keen on everything being installed directly onto the system which is what I tried to solve with Ansible-NAS. </shameless_plug>
- dragonsh 7y ago
  Docker use Dockerfile (ini) to create container images, so you need to create a soup of Dockerfile, bash scripts combine it with ansible scripts. So it is kind of incompatible, it's better to use LXD containers as I suggested in my previous post for this kind of purpose. Also if you know Docker always maintained one process one docker container so a multi-daemon system with SSH and various mail and groupware servers running is not right choice for Docker. Using docker for such things is like putting a square peg in a small round hole.
  - ptman 7y ago
    Dockerfile format is not ini. https://docs.docker.com/engine/reference/builder/ https://docs.docker.com/engine/reference/builder/ It also doesn't usually make sense to put SSH in a docker image or container, as you can enter the container using docker exec
dragonsh 7y ago
Sovereign can be used as is to create linux container with LXD without any changes. You can create container image and than use it across your immutable infrastructure. Don’t waste time in Docker, for this purpose use lxd containers [1]. LXD containers are more secure than docker in general and provide support for unprivileged containers for over 5 years. You can use your ansible scripts to create and manage your container images the way you manage bare metal or a VM. Indeed the modern LXD can help you seamlessly manage VM or Linux container [2], because when you need to directly access underlying host hardware VM is still more secure than container. Also with mount syscall interception in version 3.19 of LXD [3], it is possible to mount NFS inside container in a more secure way with unprivileged container running in user namespace. Initially docker itself was built using lxc [4], but then decided to re-invent the wheels to build its own libcontainer without any significant advantage over lxc, just NIH. Obviously given all the money which went into docker and than kubernetes using it. Docker is more famous in spite of being inferior because of the marketing money spend on it like Java (a language which is famous because, sun spend over 500 million in early years of its inception on marketing). Kubernetes is famous and a valid tool for google level of problems for 90% of startups LXD is a better fit. This is the same fight like old times when inferior technology wins due to sheer marketing like Blu-ray won but rendered not as useful, same is docker. [1] https://linuxcontainers.org/ https://linuxcontainers.org/ [2] https://discuss.linuxcontainers.org/t/lxd-3-18-has-been-released/5869 https://discuss.linuxcontainers.org/t/lxd-3-18-has-been-rele... [3] https://discuss.linuxcontainers.org/t/lxd-3-19-has-been-released/6529/2 https://discuss.linuxcontainers.org/t/lxd-3-19-has-been-rele... [4] https://vyomtech.com/2014/03/04/docker_and_linux_containers_lxc_1_0_release.html https://vyomtech.com/2014/03/04/docker_and_linux_containers_...
- jnwatson 7y ago
  Come on. I like LXD, and I used it before docker. Docker is a standard and community. You can find prebuilt containers online and layer your own additions with a single configuration line. Don’t get me wrong, Docker has made some stupid technology decisions, but the network effect of so many adopters means that it should be the default choice for any container situation.
  - 7y ago
movedx 7y ago
> In my personal experience, Docker images (and docker-compose orchestrations) are more pervasive than Ansible playbooks and when it comes to self-hosting for personal use, hosting all my services in Docker has made it much easier to deploy/maintain them Are you building every container your self? I hope so: https://blog.banyansecurity.io/blog/over-30-of-official-images-in-docker-hub-contain-high-priority-security-vulnerabilities https://blog.banyansecurity.io/blog/over-30-of-official-imag...
- jlkuester7 7y ago
  :) Yeah, that was one of the big reasons I emphasized that my setup was focused on personal use. I have decided that the potential vulnerabilities present in the official images that I use present an acceptable level of risk that I am willing to assume. Enterprise deployments and other people with different personal levels of acceptable risk will require a better (but much more time-intensive) setup.
thelittleone 7y ago
Thanks for sharing these list of FLOSS alternatives. Have you had much experience with MailU and anti spam / deliverability issues? My needs are only a few emai's a week, but previous tests with running my own mail server resulted in a high % of emails ending up in client spam folders.

tvanantwerp 7y ago

Are there any resources you recommend for learning to use Docker specifically for self-hosted services in a home lab setting?

jlkuester7 7y ago
IMHO the official Docker documentation is great! Their "getting started" page is a good overview of the Docker basics. https://docs.docker.com/get-started/ https://docs.docker.com/get-started/ The CLI --help is pretty useful too. I have also found Docker's in-depth documentation to be very helpful when trying to figure out the more complex features. I got started by just grabbing a $5 DigitalOcean droplet (can get them with Docker pre-installed) and then played around trying to setup a simple app. (I think it was RocketChat.) https://rocket.chat/docs/installation/docker-containers/ https://rocket.chat/docs/installation/docker-containers/
orionhasyou 7y ago
https://www.linuxserver.io/ https://www.linuxserver.io/ not a resource but docker containers you might want to run in your homelab. checkout awesome docker on github or katacoda

lxcfan 7y ago

The two are not incompatible. You can deploy containers with Ansible.

yjftsjthsd-h 7y ago
Indeed:) Ansible will happily interface with pretty much anything; consider that it officially supports Swarm (https://docs.ansible.com/ansible/latest/modules/docker_swarm_service_module.html https://docs.ansible.com/ansible/latest/modules/docker_swarm...), k8s (https://docs.ansible.com/ansible/latest/modules/k8s_module.html https://docs.ansible.com/ansible/latest/modules/k8s_module.h...), and puppet (https://docs.ansible.com/ansible/latest/modules/puppet_module.html https://docs.ansible.com/ansible/latest/modules/puppet_modul...) for example:) It makes excellent glue if used with other things, or a good tool all on its own. Which is especially nice because it lets you try new things without changing your overall setup too much; you can deploy all your services with RPMs, and then move one thing into docker, and still use ansible for everything:)

ggregoire 7y ago

> Not trying to start a Docker vs. Ansible flame war Don't worry, they serve very different purposes. You already probably know but Docker is for running applications in isolation, while Ansible is for provisioning and configuring hosts. For instance, you won't use Docker to harden sshd on your hosts but Ansible.

bwbmr 7y ago
Are there any good resources (besides OP) for using Ansible in a home-server environment? Especially in a mixed docker + Ansible environment.
- jefurii 7y ago
  After figuring out Ansible work I started using it to manage several personal VPS machines and the various boxes at home. I just used the Ansible docs, which are pretty good.
- S04dKHzrKT 7y ago
  https://github.com/davestephens/ansible-nas https://github.com/davestephens/ansible-nas Uses ansible + docker to setup and run a home server. Its feature-set is close to that of a typical FreeNAS setup in a home environment.
- ggregoire 7y ago
  The official doc, StackOverflow and /r/ansible are good resources imo.
- VvR-Ox 7y ago
  Just start a simple project with Ansible and have a look at the repositories using Ansible to install Docker and deploy containers on your host afterwards. You may want to put some effort (not that much) into managing your credentials with Ansible Vault[0] and you can try your playbooks e.g. on a Vagrant Machine[1] before applying them to a real host. Any tutorial will do for the beginning but you should always notice what version of Ansible you are using (vs. the one used in the tutorial) as features change and also there have been some changes to the syntax to improve readability of your playbooks. [0]: https://docs.ansible.com/ansible/latest/user_guide/vault.html https://docs.ansible.com/ansible/latest/user_guide/vault.htm... [1]: https://www.vagrantup.com/docs/provisioning/ansible.html https://www.vagrantup.com/docs/provisioning/ansible.html

newscracker 7y ago

> I would say my biggest piece of advice is to learn Docker. (Not trying to shill for Docker here The first step to learning Docker by downloading it, irritatingly, is that getting Docker Desktop on Mac or Windows requires creating a Docker Hub account and signing in! There’s even a long issue thread about this on GitHub and the response was totally irrelevant! Luckily for those who don’t want to jump through these hoops (through disposable addresses or reusable shared logins) or provide an email address, there are many people who have posted direct links to the different downloads available.

CraigJPerry 7y ago
I think you can use chocolatey to choco install it without going to the docker site.
DarthGhandi 7y ago
I remember when they changed that and the GitHub issue was one of the most hilarious examples of corporate doublespeak. "Why are you doing this?" "To improve user experience" "But now I have to sign up for an account whereas before I could just download" "Yes this will improve the experience for everyone over time"

zzzeek 7y ago

I run everything in containers and I spin up all the images / containers from ansible scripts. Am I doing it wrong? I didn't think these two tools were in conflict.

dcardoza 7y ago
They do complement one another but docker has tooling that solves similar problems. Does docker-compose (https://docs.docker.com/compose/ https://docs.docker.com/compose/) fit your needs?
- onei 7y ago
  I tend to run ansible to setup a VM, e.g. a few packages, lock it down to my taste, and then have ansible run docker-compose to deploy the application(s) of interest. You wouldn't use docker to manage the host networking (which is where ansible comes in), but packaging whatever is listening on ports as a container works really well for me
- zzzeek 7y ago
  I use ansible to build a machine from zero software installed on it to running multiple docker containers that also make use of various filesystem stuff on the host, as well as set up other non-docker things on the host like networking, selinux stuff, and crons. I like that ansible is not hardwired towards everything being a container.

davestephens 7y ago

No need for an Ansible vs. Docker argument when you can combine them both! I run a similar project called Ansible-NAS - https://github.com/davestephens/ansible-nas https://github.com/davestephens/ansible-nas - which originally came about because I fell out of love with FreeNAS, and felt I could do a better job with Ubuntu, Ansible, and a bunch of Docker images. Sovereign is awesome, I've been watching it for a while, but I'm not keen on everything being installed directly onto the system which is what I tried to solve with Ansible-NAS. </shameless_plug>

dragonsh 7y ago
Docker use Dockerfile (ini) to create container images, so you need to create a soup of Dockerfile, bash scripts combine it with ansible scripts. So it is kind of incompatible, it's better to use LXD containers as I suggested in my previous post for this kind of purpose. Also if you know Docker always maintained one process one docker container so a multi-daemon system with SSH and various mail and groupware servers running is not right choice for Docker. Using docker for such things is like putting a square peg in a small round hole.
- ptman 7y ago
  Dockerfile format is not ini. https://docs.docker.com/engine/reference/builder/ https://docs.docker.com/engine/reference/builder/ It also doesn't usually make sense to put SSH in a docker image or container, as you can enter the container using docker exec

dragonsh 7y ago

Sovereign can be used as is to create linux container with LXD without any changes. You can create container image and than use it across your immutable infrastructure. Don’t waste time in Docker, for this purpose use lxd containers [1]. LXD containers are more secure than docker in general and provide support for unprivileged containers for over 5 years. You can use your ansible scripts to create and manage your container images the way you manage bare metal or a VM. Indeed the modern LXD can help you seamlessly manage VM or Linux container [2], because when you need to directly access underlying host hardware VM is still more secure than container. Also with mount syscall interception in version 3.19 of LXD [3], it is possible to mount NFS inside container in a more secure way with unprivileged container running in user namespace. Initially docker itself was built using lxc [4], but then decided to re-invent the wheels to build its own libcontainer without any significant advantage over lxc, just NIH. Obviously given all the money which went into docker and than kubernetes using it. Docker is more famous in spite of being inferior because of the marketing money spend on it like Java (a language which is famous because, sun spend over 500 million in early years of its inception on marketing). Kubernetes is famous and a valid tool for google level of problems for 90% of startups LXD is a better fit. This is the same fight like old times when inferior technology wins due to sheer marketing like Blu-ray won but rendered not as useful, same is docker. [1] https://linuxcontainers.org/ https://linuxcontainers.org/ [2] https://discuss.linuxcontainers.org/t/lxd-3-18-has-been-released/5869 https://discuss.linuxcontainers.org/t/lxd-3-18-has-been-rele... [3] https://discuss.linuxcontainers.org/t/lxd-3-19-has-been-released/6529/2 https://discuss.linuxcontainers.org/t/lxd-3-19-has-been-rele... [4] https://vyomtech.com/2014/03/04/docker_and_linux_containers_lxc_1_0_release.html https://vyomtech.com/2014/03/04/docker_and_linux_containers_...

jnwatson 7y ago
Come on. I like LXD, and I used it before docker. Docker is a standard and community. You can find prebuilt containers online and layer your own additions with a single configuration line. Don’t get me wrong, Docker has made some stupid technology decisions, but the network effect of so many adopters means that it should be the default choice for any container situation.
- 7y ago

movedx 7y ago

> In my personal experience, Docker images (and docker-compose orchestrations) are more pervasive than Ansible playbooks and when it comes to self-hosting for personal use, hosting all my services in Docker has made it much easier to deploy/maintain them Are you building every container your self? I hope so: https://blog.banyansecurity.io/blog/over-30-of-official-images-in-docker-hub-contain-high-priority-security-vulnerabilities https://blog.banyansecurity.io/blog/over-30-of-official-imag...

jlkuester7 7y ago
:) Yeah, that was one of the big reasons I emphasized that my setup was focused on personal use. I have decided that the potential vulnerabilities present in the official images that I use present an acceptable level of risk that I am willing to assume. Enterprise deployments and other people with different personal levels of acceptable risk will require a better (but much more time-intensive) setup.

thelittleone 7y ago

Thanks for sharing these list of FLOSS alternatives. Have you had much experience with MailU and anti spam / deliverability issues? My needs are only a few emai's a week, but previous tests with running my own mail server resulted in a high % of emails ending up in client spam folders.