3 ms·

Hey! Thanks for the awesome response. Firecracker is awesome, I had a chance to use it a bit. Firecracker is a vm too, I think. It’s KVM under the hood. You are

Hey! Thanks for the awesome response. Firecracker is awesome, I had a chance to use it a bit. Firecracker is a vm too, I think. It’s KVM under the hood. You are right that in the general case going for the cloud is the most appropriate choice. I’ve made a choice to self host and it works for me in so far for a few years. I’m looking to have more automation now. What I currently have in mind is a nomad/consul (with mesh) architecture over hetzner dedicated nodes. So nomad will oversee a fleet of vms on these nodes. To use packer or similar tool to create images and deploy using nomad. This is the base and then I can add load balancers, monitoring, logging etc as needed, cloudflare etc.

oneplane 5y ago
While I know the name Hetzner I don't know if their server offering is the same as their 'cloud' offering. They do have an official Terraform provider: https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs https://registry.terraform.io/providers/hetznercloud/hcloud/... The benefit of this is that you can put all the common items in place (network, volumes, keys, firewalls) ahead of time and then just add instances as needed. Their official provider also has a lot of Datasources meaning you can automatically get all the data you need in other places and then use terraform to write it out wherever you need it. As an example, Terraform can use the hcloud provider to create or read resources and use the consul provider to then write it out to consul. Then, if your application or orchestration tools need some data they can just get it out of consul. From the looks of it, you might have the following list of actions: - Build the software from sources (whatever CI you are using) - Take the built software and put it in an image using Packer (https://www.packer.io/plugins/builders/hetzner-cloud) - Use terraform to make sure all the non-deployment-specific stuff is ready - Use something else (Harvester, Kubevirt, Nomad etc.) to make sure the right amount of servers from the correct image are started/stopped While you could technically use salt-cloud for everything, In my experience, salt is best at 'runtime' stuff, not as much preparing and desired state configuration. I personally take the approach that Terraform is for everything "outside" virtual machines/containers/FaaS, and CI/Ansible/Salt is for everything "inside" virtual machines/containers/Faas. Using a ton of different tools might make it really complicated, but trying to do everything with 'one golden tool' is going to give more problems than benefits in general. One benefit of pulling in Kubernetes (but just for the Kubevirt part) is that there is a Kubernetes Autoscaler for Hetzner which means that you can specify "I want 10 virtual machines at all times, and they have to use this specific image" and Kubernetes and the Hetzner Autoscaler will make sure that there are always 10, if one crashes it gets replaced, and when you update the image it first adds 10 replacements and then removes the 10 old ones (or you can configure a different rollover method of course). The downside is that if you don't have Kubernetes experience yet it will be a very steep learning curve and in such a case I really would not recommend this (yet). I do recommend keeping principles like GitOps (Git is the source of truth and operations have to flow through Git so you always have a record of change and can go back if needed) and disposable/cattle instances (never make an instance special, always be able to just nuke and replace it; this also means that if a system gets compromised you can just nuke/suspend it and know that you can start a new instance [with patches], and if there is an outage you know you'll be able to recover because that recovery is the same as your normal process).

oneplane 5y ago

While I know the name Hetzner I don't know if their server offering is the same as their 'cloud' offering. They do have an official Terraform provider: https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs https://registry.terraform.io/providers/hetznercloud/hcloud/... The benefit of this is that you can put all the common items in place (network, volumes, keys, firewalls) ahead of time and then just add instances as needed. Their official provider also has a lot of Datasources meaning you can automatically get all the data you need in other places and then use terraform to write it out wherever you need it. As an example, Terraform can use the hcloud provider to create or read resources and use the consul provider to then write it out to consul. Then, if your application or orchestration tools need some data they can just get it out of consul. From the looks of it, you might have the following list of actions: - Build the software from sources (whatever CI you are using) - Take the built software and put it in an image using Packer (https://www.packer.io/plugins/builders/hetzner-cloud) - Use terraform to make sure all the non-deployment-specific stuff is ready - Use something else (Harvester, Kubevirt, Nomad etc.) to make sure the right amount of servers from the correct image are started/stopped While you could technically use salt-cloud for everything, In my experience, salt is best at 'runtime' stuff, not as much preparing and desired state configuration. I personally take the approach that Terraform is for everything "outside" virtual machines/containers/FaaS, and CI/Ansible/Salt is for everything "inside" virtual machines/containers/Faas. Using a ton of different tools might make it really complicated, but trying to do everything with 'one golden tool' is going to give more problems than benefits in general. One benefit of pulling in Kubernetes (but just for the Kubevirt part) is that there is a Kubernetes Autoscaler for Hetzner which means that you can specify "I want 10 virtual machines at all times, and they have to use this specific image" and Kubernetes and the Hetzner Autoscaler will make sure that there are always 10, if one crashes it gets replaced, and when you update the image it first adds 10 replacements and then removes the 10 old ones (or you can configure a different rollover method of course). The downside is that if you don't have Kubernetes experience yet it will be a very steep learning curve and in such a case I really would not recommend this (yet). I do recommend keeping principles like GitOps (Git is the source of truth and operations have to flow through Git so you always have a record of change and can go back if needed) and disposable/cattle instances (never make an instance special, always be able to just nuke and replace it; this also means that if a system gets compromised you can just nuke/suspend it and know that you can start a new instance [with patches], and if there is an outage you know you'll be able to recover because that recovery is the same as your normal process).