3 ms·

I have attempted this using https://coreos.com/kubernetes/docs/latest/kubernetes-on-baremetal.html https://coreos.com/kubernetes/docs/latest/kubernetes-on-bare.

I have attempted this using https://coreos.com/kubernetes/docs/latest/kubernetes-on-baremetal.html https://coreos.com/kubernetes/docs/latest/kubernetes-on-bare... . One can get quite far, but I am not sure it is worth the effort. I understand this is a vague analogy, but it feels like you're trying to setup email infrastructure when what you really want is to send email. It is a lot easier on the public cloud, or easier still on a managed service.

iagooar 10y ago
What about people who cannot go to the cloud? What about people who need more performance? Lower costs? Kubernetes seems to be such a great abstraction for the underlying hardware, why not use it where it is needed the most?
- jdoliner 10y ago
  People do run Kubernetes on bare metal... but there's no denying it takes effort. That effort should be coming down over time, but unfortunately there's still a ways to go.
  - AlexB138 10y ago
    That's not entirely true. I attend meet-ups with engineers from a fairly large company whose name you would recognize who run K8s on bare metal. I know that's vague, but I don't know how public their infrastructure is, so I'd rather not speak out of turn. You're absolutely right that it takes some effort.
    - SEJeff 10y ago
      Bloomberg runs Kubernetes on bare metal and one of my friends is an engineer on the project (but did not speak at KubeCon about it). They've got a relatively sizable install as well.
- nixgeek 10y ago
  citation required, since there are IaaS offerings that do not involve a hypervisor (e.g. Rackspace OnMetal) and therefore its possible to move "to the cloud" without performance penalties. on costs via Reserved Instances and negotiating with your account team at larger volume, it can be pretty reasonable, and for a smaller outfit, substantially cheaper than hiring experts in running physical infrastructure of your own.
  - manigandham 10y ago
    Citation needed for what? its common knowledge that the major clouds VMs with lower performance and higher cost in exchange for flexibility.
  - vidarh 10y ago
    > and for a smaller outfit, substantially cheaper than hiring experts in running physical infrastructure of your own. You needs to have the skillsets to manage cloud deployments too. For the systems I manage, which range from actual bare metal, via dedicated servers, to VPSs and AWS deployments, the incremental effort spent on managing hardware as you go down towards bare metal tends to be pretty much a rounding error compard to the overall operations effort. Once the systems have been wired up, and PXE booted into a suitable setup, the effort is pretty much the same. And with the cost differential, I'd say once you go over a few hundred dollars a month on servers that stay up 24/7, you're losing money on public cloud deployments vs. managed dedicated hosting. Once you go over $1k to $2k/month, you're losing money vs. colo. For people who actually have a lot of batch jobs where servers stay up for less than 6-8 hours a day, the maths look different, but it's very rare I come across cloud setups that are cheaper than dedicated, with all staff costs etc. accounted for.
  - iagooar 10y ago
    citation required for what? For stating, that it is complicated to install on bare-metal and that it's been a priority for Kubernetes to be deploying on cloud servers?
jabzd 10y ago
I felt very similar. Especially thinking about it from a disaster recovery/re-bootstrapping perspective. That scared me into using docker swarm kit which is stupidly easy to bootstrap. Keep in mind that's not necessarily a like for like replacement but it was enough for my use case. I've been keeping an eye out waiting to feel more comfortable to use and support a kube cluster on bare metal.
philips 10y ago
What would your ideal setup look like on baremetal? The Kubernetes on baremetal setup above is for full automation. You can do simpler manual installs with bootkube[1] or kubeadm[2]; and this will further simplifying over the next few months. [1] https://github.com/kubernetes-incubator/bootkube https://github.com/kubernetes-incubator/bootkube [2] http://kubernetes.io/docs/getting-started-guides/kubeadm/ http://kubernetes.io/docs/getting-started-guides/kubeadm/

iagooar 10y ago

What about people who cannot go to the cloud? What about people who need more performance? Lower costs? Kubernetes seems to be such a great abstraction for the underlying hardware, why not use it where it is needed the most?

jdoliner 10y ago
People do run Kubernetes on bare metal... but there's no denying it takes effort. That effort should be coming down over time, but unfortunately there's still a ways to go.
- AlexB138 10y ago
  That's not entirely true. I attend meet-ups with engineers from a fairly large company whose name you would recognize who run K8s on bare metal. I know that's vague, but I don't know how public their infrastructure is, so I'd rather not speak out of turn. You're absolutely right that it takes some effort.
  - SEJeff 10y ago
    Bloomberg runs Kubernetes on bare metal and one of my friends is an engineer on the project (but did not speak at KubeCon about it). They've got a relatively sizable install as well.
nixgeek 10y ago
citation required, since there are IaaS offerings that do not involve a hypervisor (e.g. Rackspace OnMetal) and therefore its possible to move "to the cloud" without performance penalties. on costs via Reserved Instances and negotiating with your account team at larger volume, it can be pretty reasonable, and for a smaller outfit, substantially cheaper than hiring experts in running physical infrastructure of your own.
- manigandham 10y ago
  Citation needed for what? its common knowledge that the major clouds VMs with lower performance and higher cost in exchange for flexibility.
- vidarh 10y ago
  > and for a smaller outfit, substantially cheaper than hiring experts in running physical infrastructure of your own. You needs to have the skillsets to manage cloud deployments too. For the systems I manage, which range from actual bare metal, via dedicated servers, to VPSs and AWS deployments, the incremental effort spent on managing hardware as you go down towards bare metal tends to be pretty much a rounding error compard to the overall operations effort. Once the systems have been wired up, and PXE booted into a suitable setup, the effort is pretty much the same. And with the cost differential, I'd say once you go over a few hundred dollars a month on servers that stay up 24/7, you're losing money on public cloud deployments vs. managed dedicated hosting. Once you go over $1k to $2k/month, you're losing money vs. colo. For people who actually have a lot of batch jobs where servers stay up for less than 6-8 hours a day, the maths look different, but it's very rare I come across cloud setups that are cheaper than dedicated, with all staff costs etc. accounted for.
- iagooar 10y ago
  citation required for what? For stating, that it is complicated to install on bare-metal and that it's been a priority for Kubernetes to be deploying on cloud servers?

jabzd 10y ago

I felt very similar. Especially thinking about it from a disaster recovery/re-bootstrapping perspective. That scared me into using docker swarm kit which is stupidly easy to bootstrap. Keep in mind that's not necessarily a like for like replacement but it was enough for my use case. I've been keeping an eye out waiting to feel more comfortable to use and support a kube cluster on bare metal.

philips 10y ago

What would your ideal setup look like on baremetal? The Kubernetes on baremetal setup above is for full automation. You can do simpler manual installs with bootkube[1] or kubeadm[2]; and this will further simplifying over the next few months. [1] https://github.com/kubernetes-incubator/bootkube https://github.com/kubernetes-incubator/bootkube [2] http://kubernetes.io/docs/getting-started-guides/kubeadm/ http://kubernetes.io/docs/getting-started-guides/kubeadm/