3 ms·

At my previous job we ran a bare metal cluster at Hetzner, and monitoring the hardware was quite an intensive task. Always monitoring hard drives network bottle

by tinco 5y ago

At my previous job we ran a bare metal cluster at Hetzner, and monitoring the hardware was quite an intensive task. Always monitoring hard drives network bottlenecks, CPU usage, etc. This was before K8s, so it might not be comparable to today. Would you say bare metal cost a lot of extra monitoring/maintenance, or is this something you did on the cloud hardware as well anyway? Do you run virtualization on the Hetzner machines?

hardwaresofton 5y ago
Just a note, there are some companies that have been popping up recently to try to bridge the gap for services on clouds like Hetzner. https://elest.io https://elest.io https://nimbusws.com https://nimbusws.com (I'm building this one so I'm biased for it). > Would you say bare metal cost a lot of extra monitoring/maintenance, or is this something you did on the cloud hardware as well anyway? Do you run virtualization on the Hetzner machines? It cost a lot of monitoring/maintenance up front, but once things are purring the costs amortize really well. Hetzner has the occasional hard drive failure[0], but you should be running in a RAIDed setup (that's the default for Hetzner-installed OSes), so you do have some time. They also replace drives very quickly. If you really want to remove this headache, you run something like Ceph and make sure data copies are properly replicated to multiple hosts and you'll be fine if two drives on a single host die at the same time. Obviously nothing is every that easy but I know that I spend pretty much no time thinking about it these days. I run a Kubernetes cluster and most of my downtime/outages have been self-inflicted, I'm wonderfully happy with my cluster now. Also another thing to note is that control plane downtime != workload downtime, which is another nice thing -- and you can hook up grafana etc to monitor it all. [0]: https://vadosware.io/post/handling-your-first-dead-hetzner-hard-drive/ https://vadosware.io/post/handling-your-first-dead-hetzner-h...
vidarh 5y ago
I would say cloud costs a lot of extra maintenance. When I did contracting, those of my clients who insisted on AWS tended to be far more profitable clients for me, because they needed help with so much more stuff. In terms of monitoring, it boils down to picking a solution and building the appropriate monitoring agent into your deployment. I've run basically anything I run in some virtualized env. or other since ~2006 at least, be it OpenVz (ages ago), KVM, or docker. And that goes for Hetzner too. It makes it easy to ensure that the app environment is identical no matter where you move things. I managed on environment where we had stuff running on prem and in several colo's, on dedicated servers at Hetzner, and in VMs, and even on the VMs we still containerised everything - deployment of new containers was identical no matter where you deployed. All of the environment specific details were hidden in the initial host setup.

by tinco 5y ago

hardwaresofton 5y ago

Just a note, there are some companies that have been popping up recently to try to bridge the gap for services on clouds like Hetzner. https://elest.io https://elest.io https://nimbusws.com https://nimbusws.com (I'm building this one so I'm biased for it). > Would you say bare metal cost a lot of extra monitoring/maintenance, or is this something you did on the cloud hardware as well anyway? Do you run virtualization on the Hetzner machines? It cost a lot of monitoring/maintenance up front, but once things are purring the costs amortize really well. Hetzner has the occasional hard drive failure[0], but you should be running in a RAIDed setup (that's the default for Hetzner-installed OSes), so you do have some time. They also replace drives very quickly. If you really want to remove this headache, you run something like Ceph and make sure data copies are properly replicated to multiple hosts and you'll be fine if two drives on a single host die at the same time. Obviously nothing is every that easy but I know that I spend pretty much no time thinking about it these days. I run a Kubernetes cluster and most of my downtime/outages have been self-inflicted, I'm wonderfully happy with my cluster now. Also another thing to note is that control plane downtime != workload downtime, which is another nice thing -- and you can hook up grafana etc to monitor it all. [0]: https://vadosware.io/post/handling-your-first-dead-hetzner-hard-drive/ https://vadosware.io/post/handling-your-first-dead-hetzner-h...

vidarh 5y ago

I would say cloud costs a lot of extra maintenance. When I did contracting, those of my clients who insisted on AWS tended to be far more profitable clients for me, because they needed help with so much more stuff. In terms of monitoring, it boils down to picking a solution and building the appropriate monitoring agent into your deployment. I've run basically anything I run in some virtualized env. or other since ~2006 at least, be it OpenVz (ages ago), KVM, or docker. And that goes for Hetzner too. It makes it easy to ensure that the app environment is identical no matter where you move things. I managed on environment where we had stuff running on prem and in several colo's, on dedicated servers at Hetzner, and in VMs, and even on the VMs we still containerised everything - deployment of new containers was identical no matter where you deployed. All of the environment specific details were hidden in the initial host setup.