8 ms·

Kubernetes – The Hard Way

157 points by yarapavan 10y ago

okket 10y ago
Previous discussion: https://news.ycombinator.com/item?id=12323187 https://news.ycombinator.com/item?id=12323187 (23 days ago, 79 comments)
- yarapavan 10y ago
  Thanks! Please see author's tweet here - https://twitter.com/kelseyhightower/status/774983770956898307 https://twitter.com/kelseyhightower/status/77498377095689830... The github repo is updated with updated DNS add-on, better examples and AWS support.
  - justinsb 10y ago
    Please note, this is not a supported AWS configuration. This is just a way to run Kubernetes on AWS instances, but it is really just treating them as bare-metal. My advice: You should be using kops or kube-aws for production. (I work on kops, so I am biased to believe it is important)
    - yarapavan 10y ago
      Thank you! Learned about kops today.
brobinson 10y ago
Is there a "Kubernetes - The non-Google Way"? It seems like every tutorial is for GCE.
- crb 10y ago
  The likely reason for this being posted to HN again today is that Kelsey added AWS instructions along the GCE ones.
  - brobinson 10y ago
    Oh, I only scanned the top of the README and it looked the same as the last time it was posted. Thanks for the pointer!
- thallian 10y ago
  There is a step by step guide from CoreOS: https://coreos.com/kubernetes/docs/latest/getting-started.html https://coreos.com/kubernetes/docs/latest/getting-started.ht... Never tried it though, so I can't speak of the quality (and of course it talks about running it on CoreOS).
  - brobinson 10y ago
    Thanks! I'll give it a shot!
- kelseyhightower 10y ago
  I've updated the tutorial to include AWS a few days. Based on your feedback I've added a note regarding AWS support toward the top of the project README and in the project description field. In some cases you'll find side-by-side labs that focus solely on AWS like this one for bootstrapping the underlying compute: https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/01-infrastructure-aws.md https://github.com/kelseyhightower/kubernetes-the-hard-way/b... In other parts of the tutorial you'll find sections for both GCE and AWS that highlight the different commands required for each platform - only a small fraction of the tutorial requires something different between GCE and AWS. Both cloud providers are now treated equally in the updated tutorial.
  - brobinson 10y ago
    Thanks, Kelsey!
- jschneiderhan 10y ago
  I came across an AWS version while googling the other day. I figured I'd share in case that is more useful to you. https://github.com/ivx/kubernetes-the-hard-way-aws https://github.com/ivx/kubernetes-the-hard-way-aws
wyclif 10y ago
The very first Lab link gives a 404 error. Heads up.
- gdubya 10y ago
  That's why it's "the hard way"
- t0mek 10y ago
  Open the README.md file directly. Links will work then. https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/README.md https://github.com/kelseyhightower/kubernetes-the-hard-way/b... Apparently, there's something odd in the GitHub support for relative links in the main project README.md.
- kelseyhightower 10y ago
  Thanks for mentioning this. It looks like there is something wrong on GitHub's end. The current, temporary, workaround is to visit the README directly and the relative links should work.
- kelseyhightower 10y ago
  I was able to contact GitHub support and reported the issue. Looks like relative links from the main page are working again. Thanks for the heads up.
cies 10y ago
Cannot find much on mounting storage to the container. It exactly that --storage-- that has been recognized as a difficult problem when moving an infrastructure to containers. Given the title "The Hard Way", I kind of expected it would dive deeper into this topic.
- jordanthoms 10y ago
  Of course - following the first law of technical documentation, the amount written about it is inversely proportional to the complexity of the problem. Seriously though, it is getting a bit silly how almost every single kubernetes demo involving persistent data just uses emptyDir or hostPath and we are all supposed to pretend that those are a production-ready solution that won't lose data. AFAIK the PersistentVolumeClaim and StorageClass stuff will be the solution for this?
  - cies 10y ago
    To me it seems that there are many approaches. A storage cluster seems to be a necessity for anything serious. But then the questions is: which one? Open source solutions are not too many (Gluster, Ceph, FreeNAS come to mind), and from the closed source camp ScaleIO looks really promising. But non of these are part of your K8s tutorial :)
    - creshal 10y ago
      And gluster at least has horrible performance and stability issues. And with containers, kernel panics kill an entire host…
      - strikerz 10y ago
        Do you have more details on performance and stability issues? The recent releases of gluster have been working very well. Gluster is completely in userspace. How would stability issues, if any, in userspace cause kernel panics? Can you comment more about that?
rmoriz 10y ago
I'm surprised that RedHat's OpenShift 3 / Origin does not get any publicity on HN. It uses Kubernetes (of which RedHat silently became the largest committer) and also fills several missing parts including a CD pipeline. https://www.openshift.org/ https://www.openshift.org/ It's amazing to see how many large Enterprises already selected OpenShift, either in various cloud setups or on premise scenarios. Looks like that for many HN people there is only AWS and GCE out there.
- pjmlp 10y ago
  And Azure.
- ogrisel 10y ago
  Apparently the goal of this guide it to setup a kubernetes from scratch (on top of VMs provisioned on an orchestration-agnostic public cloud), not how to use a managed kubernetes cluster as provided by openshift.
  - avtar 10y ago
    Perhaps the parent was referring to the self-hosting OpenShift option which acts as a Kubernetes distribution: https://docs.openshift.org/latest/getting_started/administrators.html#getting-started-administrators https://docs.openshift.org/latest/getting_started/administra...
- deleted 10y ago
  [deleted]
- moe 10y ago
  Many "large enterprises" have also selected Windows as their server OS and Oracle for their database... After decades of replacing RHEL where possible (and plenty agony where it wasn't possible) I can not help but chuckle at the idea of even considering the company that brought us 'yum' for a virtualization platform.
  - otterley 10y ago
justinsb 10y ago
Please note that this guide is _only_ for learning purposes - it makes a lot of decisions that do not follow the recommended production approach. If you're installing for production purposes, you should use one of the existing tools if you can (I work on kops and so naturally recommend it for AWS). If none of the tools meet your needs, you should probably first open issues on those tools; if you still must write your own installation method, you should combine this guide with the official documentation to understand a cluster as created by one of the recommended tools, before building your own.
- kelseyhightower 10y ago
  Happy to improve the tutorial to incorporate the production approach that should be taken for each task. The main goal is allow people to run through a cluster bootstrap step by step so they learn how things work. If you don't mind filing a few issues on the project I'll be happy to rework each lab to start addressing these concerns. Until then I've added a note to the README regarding production readiness, but that should not prevent people from learning.
  - justinsb 10y ago
    You should participate in sig-cluster-lifecycle. The conclusion of that sig (or my interpretation of it) was that having a production suitable cluster is - today - essentially orthogonal to having a discoverable build-your own cluster. So we are all best served by having this document for those that want to learn about how kubernetes works, but a separate approach for installing a production cluster that prioritizes correctness over obviousness. I think what you are doing is great, but I worry that using words like "support" imply that you will be offering some support for people that try to run this in production. However, the long term goal of sig-cluster-lifecycle is to bridge the gap here, to make a production configuration easy and obvious. So I'd love to see you start using the new kubeadm work, for example, so that we can meet in the middle!
    - kelseyhightower 10y ago
      Thanks for the feedback. My goal is to augment those tools with step-by-step documentation so people get an opportunity to learn how things work. Yes, I will attempt to "support" people as they attempt to learn things the hard way. I would also like to note that I'm not in competition with those automation tools. I'm just really focused on helping people learn. Some parts of the Kubernetes community really want to learn this stuff without an automation tool so they can skill up to build their own, custom, tools that match their preferences and tradeoffs.

157 points by yarapavan 10y ago

okket 10y ago

Previous discussion: https://news.ycombinator.com/item?id=12323187 https://news.ycombinator.com/item?id=12323187 (23 days ago, 79 comments)

yarapavan 10y ago
Thanks! Please see author's tweet here - https://twitter.com/kelseyhightower/status/774983770956898307 https://twitter.com/kelseyhightower/status/77498377095689830... The github repo is updated with updated DNS add-on, better examples and AWS support.
- justinsb 10y ago
  Please note, this is not a supported AWS configuration. This is just a way to run Kubernetes on AWS instances, but it is really just treating them as bare-metal. My advice: You should be using kops or kube-aws for production. (I work on kops, so I am biased to believe it is important)
  - yarapavan 10y ago
    Thank you! Learned about kops today.

brobinson 10y ago

Is there a "Kubernetes - The non-Google Way"? It seems like every tutorial is for GCE.

crb 10y ago
The likely reason for this being posted to HN again today is that Kelsey added AWS instructions along the GCE ones.
- brobinson 10y ago
  Oh, I only scanned the top of the README and it looked the same as the last time it was posted. Thanks for the pointer!
thallian 10y ago
There is a step by step guide from CoreOS: https://coreos.com/kubernetes/docs/latest/getting-started.html https://coreos.com/kubernetes/docs/latest/getting-started.ht... Never tried it though, so I can't speak of the quality (and of course it talks about running it on CoreOS).
- brobinson 10y ago
  Thanks! I'll give it a shot!
kelseyhightower 10y ago
I've updated the tutorial to include AWS a few days. Based on your feedback I've added a note regarding AWS support toward the top of the project README and in the project description field. In some cases you'll find side-by-side labs that focus solely on AWS like this one for bootstrapping the underlying compute: https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/01-infrastructure-aws.md https://github.com/kelseyhightower/kubernetes-the-hard-way/b... In other parts of the tutorial you'll find sections for both GCE and AWS that highlight the different commands required for each platform - only a small fraction of the tutorial requires something different between GCE and AWS. Both cloud providers are now treated equally in the updated tutorial.
- brobinson 10y ago
  Thanks, Kelsey!
jschneiderhan 10y ago
I came across an AWS version while googling the other day. I figured I'd share in case that is more useful to you. https://github.com/ivx/kubernetes-the-hard-way-aws https://github.com/ivx/kubernetes-the-hard-way-aws

wyclif 10y ago

The very first Lab link gives a 404 error. Heads up.

gdubya 10y ago
That's why it's "the hard way"
t0mek 10y ago
Open the README.md file directly. Links will work then. https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/README.md https://github.com/kelseyhightower/kubernetes-the-hard-way/b... Apparently, there's something odd in the GitHub support for relative links in the main project README.md.
kelseyhightower 10y ago
Thanks for mentioning this. It looks like there is something wrong on GitHub's end. The current, temporary, workaround is to visit the README directly and the relative links should work.
kelseyhightower 10y ago
I was able to contact GitHub support and reported the issue. Looks like relative links from the main page are working again. Thanks for the heads up.

cies 10y ago

Cannot find much on mounting storage to the container. It exactly that --storage-- that has been recognized as a difficult problem when moving an infrastructure to containers. Given the title "The Hard Way", I kind of expected it would dive deeper into this topic.

jordanthoms 10y ago
Of course - following the first law of technical documentation, the amount written about it is inversely proportional to the complexity of the problem. Seriously though, it is getting a bit silly how almost every single kubernetes demo involving persistent data just uses emptyDir or hostPath and we are all supposed to pretend that those are a production-ready solution that won't lose data. AFAIK the PersistentVolumeClaim and StorageClass stuff will be the solution for this?
- cies 10y ago
  To me it seems that there are many approaches. A storage cluster seems to be a necessity for anything serious. But then the questions is: which one? Open source solutions are not too many (Gluster, Ceph, FreeNAS come to mind), and from the closed source camp ScaleIO looks really promising. But non of these are part of your K8s tutorial :)
  - creshal 10y ago
    And gluster at least has horrible performance and stability issues. And with containers, kernel panics kill an entire host…
    - strikerz 10y ago
      Do you have more details on performance and stability issues? The recent releases of gluster have been working very well. Gluster is completely in userspace. How would stability issues, if any, in userspace cause kernel panics? Can you comment more about that?

rmoriz 10y ago

I'm surprised that RedHat's OpenShift 3 / Origin does not get any publicity on HN. It uses Kubernetes (of which RedHat silently became the largest committer) and also fills several missing parts including a CD pipeline. https://www.openshift.org/ https://www.openshift.org/ It's amazing to see how many large Enterprises already selected OpenShift, either in various cloud setups or on premise scenarios. Looks like that for many HN people there is only AWS and GCE out there.

pjmlp 10y ago
And Azure.
ogrisel 10y ago
Apparently the goal of this guide it to setup a kubernetes from scratch (on top of VMs provisioned on an orchestration-agnostic public cloud), not how to use a managed kubernetes cluster as provided by openshift.
- avtar 10y ago
  Perhaps the parent was referring to the self-hosting OpenShift option which acts as a Kubernetes distribution: https://docs.openshift.org/latest/getting_started/administrators.html#getting-started-administrators https://docs.openshift.org/latest/getting_started/administra...
deleted 10y ago
[deleted]
moe 10y ago
Many "large enterprises" have also selected Windows as their server OS and Oracle for their database... After decades of replacing RHEL where possible (and plenty agony where it wasn't possible) I can not help but chuckle at the idea of even considering the company that brought us 'yum' for a virtualization platform.
- otterley 10y ago

justinsb 10y ago

Please note that this guide is _only_ for learning purposes - it makes a lot of decisions that do not follow the recommended production approach. If you're installing for production purposes, you should use one of the existing tools if you can (I work on kops and so naturally recommend it for AWS). If none of the tools meet your needs, you should probably first open issues on those tools; if you still must write your own installation method, you should combine this guide with the official documentation to understand a cluster as created by one of the recommended tools, before building your own.

kelseyhightower 10y ago
Happy to improve the tutorial to incorporate the production approach that should be taken for each task. The main goal is allow people to run through a cluster bootstrap step by step so they learn how things work. If you don't mind filing a few issues on the project I'll be happy to rework each lab to start addressing these concerns. Until then I've added a note to the README regarding production readiness, but that should not prevent people from learning.
- justinsb 10y ago
  You should participate in sig-cluster-lifecycle. The conclusion of that sig (or my interpretation of it) was that having a production suitable cluster is - today - essentially orthogonal to having a discoverable build-your own cluster. So we are all best served by having this document for those that want to learn about how kubernetes works, but a separate approach for installing a production cluster that prioritizes correctness over obviousness. I think what you are doing is great, but I worry that using words like "support" imply that you will be offering some support for people that try to run this in production. However, the long term goal of sig-cluster-lifecycle is to bridge the gap here, to make a production configuration easy and obvious. So I'd love to see you start using the new kubeadm work, for example, so that we can meet in the middle!
  - kelseyhightower 10y ago
    Thanks for the feedback. My goal is to augment those tools with step-by-step documentation so people get an opportunity to learn how things work. Yes, I will attempt to "support" people as they attempt to learn things the hard way. I would also like to note that I'm not in competition with those automation tools. I'm just really focused on helping people learn. Some parts of the Kubernetes community really want to learn this stuff without an automation tool so they can skill up to build their own, custom, tools that match their preferences and tradeoffs.