5 ms·

> I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. Interesting. Do

by jcynix 4y ago

> I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. Interesting. Do you have any links to howtos or beginner's guides to such a setup?

BrandoElFollito 4y ago
I do not have a documentation, but I do disaster recovery tests from time to time. This is how you can try it out: - download the ISO of a linux distribution, Arch is good because you have continuous updates (there is no "version") - start it on a VM engine (VirtualBox, Hyper-V on Windows, VMWare, ...) - from that point on - START DOCUMENTING - try to on docker install a program you know that is not too complex network-wise or just start with "hello-world" (https://hub.docker.com/_/hello-world https://hub.docker.com/_/hello-world) - you will find that when running "docker pull hello-world", docker is not installed - install docker on Arch according to Arch docs. DOCUMENT that step - retry, hello-world works - now try something like https://github.com/mendhak/docker-http-https-echo https://github.com/mendhak/docker-http-https-echo - you will learn the basics of docker networking, read some docs or just try until you have a curl call working - at that point you can try a program you know (nextcloud, syncting, ...), pulling it from docker hub and make it work. pay attention to two things: the network and the persistent volumes (I recommend, at least for the start, the file-based ones, not the docker ones) - grab a beer, you are 90% done, good work - have a close look at Caddy - this is a web server similar to apache, nginx but MUCH much better. So much better that I have no words. - you will use it as a proxy server for your containers, so that you can get to them via https://nextcloud.yourdomain.com https://nextcloud.yourdomain.com. It os worthwhile to get your domain even if you do not expose anything because things are much easier that way (caddy will manage the TLS part) - now learn docker-compose and add all your dockers to it (it is a YAML description of your containers). - add backup, this will be easier if you add this program on the OS itself (it can sure be in a container but I preferred having that part independent). I recommend Borg despite its few poor choices in the design (that are not likely to bite you at that point) TADAM! you are done. You are independent of the OS, if you want to install fedora or whatever it just doe snot matter because i) all your programs are maintained by someone else (than you to the maintainers, it is nice to donate sometimes) and ii) your backup is data that is easily pluggable back to a ne instance of the software Testing new programs is super easy (you just add them to the doclker compose YAML). I truly recommend you try with a VM and you will quickly realize it is time to reformat your server and put everything under docker :) AMA if you have questions. PS. The documentation you wrote (likely 10 lines or so) is now your DRP, you can test it from time to time just to make sure you can recover everything (= the data) from the backup
- doubled112 4y ago
  Docker and docker-compose also make it really easy to store all of the data in a single location, which opens up easy snapshot management with something like ZFS or Btrfs, and backups that way. This is a thing I've found throws people off - you don't need the OS at all, just those volumes. /srv/docker_data name_of_project/ docker-compose.yml volumes/ my_project_db1/ Just for example. At upgrade time I can snapshot that whole project and test, and revert if it goes bad.
  - BrandoElFollito 4y ago
    Your file structure is interesting, I like the fact that the data and composition are together. I actually went away from separate docker-compose files and use a large one (and wrote a small utility to start, stop, pull, etc.) I will have a deeper look at that, if I managed to combine this with the caddy configurations that would be pretty much awesome :) EDIT: caddy supports globs, this si won-der-ful. I will switch to your configuration and rewrite my tool accordingly (adding a way to bootstrap the docker-compose file and the caddy config). Thank for having chimed in!
    - rovr138 4y ago
      Group things logically vs just doing one huge docker-compose file or 500 individual things. If you need 7 services together for that feature to work right, write them together. If you need 3 for the next one, group those. If you at some point want to remove one, just bring down that docker-compose. That's it. Up to you to then backup/delete/leave as is. You can write a script to automate when you start the machine. But you can also set things that are needed to autostart.
      - doubled112 4y ago
        This is it. It gives me control of each application individually. I can snapshot, back up, and even migrate each application between machines easily. It also helps out when I have an application that I need to control versions in. Using the latest tag can be fun and exciting (read dangerous and cause breakage), so if the tag is in the docker-compose file I immediately know what version that snapshot was running if I need to look back.

by jcynix 4y ago

BrandoElFollito 4y ago

I do not have a documentation, but I do disaster recovery tests from time to time. This is how you can try it out: - download the ISO of a linux distribution, Arch is good because you have continuous updates (there is no "version") - start it on a VM engine (VirtualBox, Hyper-V on Windows, VMWare, ...) - from that point on - START DOCUMENTING - try to on docker install a program you know that is not too complex network-wise or just start with "hello-world" (https://hub.docker.com/_/hello-world https://hub.docker.com/_/hello-world) - you will find that when running "docker pull hello-world", docker is not installed - install docker on Arch according to Arch docs. DOCUMENT that step - retry, hello-world works - now try something like https://github.com/mendhak/docker-http-https-echo https://github.com/mendhak/docker-http-https-echo - you will learn the basics of docker networking, read some docs or just try until you have a curl call working - at that point you can try a program you know (nextcloud, syncting, ...), pulling it from docker hub and make it work. pay attention to two things: the network and the persistent volumes (I recommend, at least for the start, the file-based ones, not the docker ones) - grab a beer, you are 90% done, good work - have a close look at Caddy - this is a web server similar to apache, nginx but MUCH much better. So much better that I have no words. - you will use it as a proxy server for your containers, so that you can get to them via https://nextcloud.yourdomain.com https://nextcloud.yourdomain.com. It os worthwhile to get your domain even if you do not expose anything because things are much easier that way (caddy will manage the TLS part) - now learn docker-compose and add all your dockers to it (it is a YAML description of your containers). - add backup, this will be easier if you add this program on the OS itself (it can sure be in a container but I preferred having that part independent). I recommend Borg despite its few poor choices in the design (that are not likely to bite you at that point) TADAM! you are done. You are independent of the OS, if you want to install fedora or whatever it just doe snot matter because i) all your programs are maintained by someone else (than you to the maintainers, it is nice to donate sometimes) and ii) your backup is data that is easily pluggable back to a ne instance of the software Testing new programs is super easy (you just add them to the doclker compose YAML). I truly recommend you try with a VM and you will quickly realize it is time to reformat your server and put everything under docker :) AMA if you have questions. PS. The documentation you wrote (likely 10 lines or so) is now your DRP, you can test it from time to time just to make sure you can recover everything (= the data) from the backup

doubled112 4y ago
Docker and docker-compose also make it really easy to store all of the data in a single location, which opens up easy snapshot management with something like ZFS or Btrfs, and backups that way. This is a thing I've found throws people off - you don't need the OS at all, just those volumes. /srv/docker_data name_of_project/ docker-compose.yml volumes/ my_project_db1/ Just for example. At upgrade time I can snapshot that whole project and test, and revert if it goes bad.
- BrandoElFollito 4y ago
  Your file structure is interesting, I like the fact that the data and composition are together. I actually went away from separate docker-compose files and use a large one (and wrote a small utility to start, stop, pull, etc.) I will have a deeper look at that, if I managed to combine this with the caddy configurations that would be pretty much awesome :) EDIT: caddy supports globs, this si won-der-ful. I will switch to your configuration and rewrite my tool accordingly (adding a way to bootstrap the docker-compose file and the caddy config). Thank for having chimed in!
  - rovr138 4y ago
    Group things logically vs just doing one huge docker-compose file or 500 individual things. If you need 7 services together for that feature to work right, write them together. If you need 3 for the next one, group those. If you at some point want to remove one, just bring down that docker-compose. That's it. Up to you to then backup/delete/leave as is. You can write a script to automate when you start the machine. But you can also set things that are needed to autostart.
    - doubled112 4y ago
      This is it. It gives me control of each application individually. I can snapshot, back up, and even migrate each application between machines easily. It also helps out when I have an application that I need to control versions in. Using the latest tag can be fun and exciting (read dangerous and cause breakage), so if the tag is in the docker-compose file I immediately know what version that snapshot was running if I need to look back.