16 ms·

Kubernetes 1.10 released

318 points by el_duderino 9y ago

BaconJuice 9y ago
Hi..can someone ELI5 to me what Kubernetes is? Also what's the best way to get started/tutorials you can recommend for a new user? Thank you!
- thinkingemote 9y ago
  Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications
  - mohaine 9y ago
    It is also mostly cloud agnostic which is nice. This gives a degree of cross cloud portability to application suites. The setup of the cluster is cloud specific but after that everything can just talk Kubernetes.
- raker 9y ago
  The simplest way to explain it is that it is google's version of docker swarm. Aside from that, it's a way to manage the deployment of containers across one or more hosts. Getting more traffic than usual? You have a program that detects it and sends a command to kubernetes/dockerswarm that tells it to scale up the number of web containers. Stuff like that.
- zaphar 9y ago
  Kubernetes orchestrates clusters of containers in a data center. It chooses where to run the containers, wires up networking between them. And keeps them running. I'm on my phone so I can't grab the link for you but a good intro is Kelsey Hightowers Kubernetes The Hard Way on GitHub.
- bronak1 9y ago
  Its a tool for DevOps helping orchestrate container deployments and scaling, you probably want to play around with docker before diving into to kubernetes.
- peterwwillis 9y ago
  Kubernetes is a thing that makes it easier for one program to talk to several other programs, by tying them together with elastic string, and putting them in whatever room it thinks they should live, and makes them as big or small as they said they needed to be. It runs in the engine bay of the truck, not in the cab of the truck, or in the container.
- ams6110 9y ago
  It's the latest thing that everyone is excited about and nobody will be interested in next year. Same boat as docker, openstack, mesos, et. al.
  - peterwwillis 9y ago
    Most of these tools have been around for more than 5 years, many organizations have invested tons of money to use them, and each is supported financially by one or more corporations, so by that metric alone they aren't going anywhere anytime soon.
  - deleted 9y ago
    [deleted]
- strathos 9y ago
  Katacoda has excellent tutorials/labs for Kubernetes. https://www.katacoda.com/learn https://www.katacoda.com/learn
- peeyek 9y ago
  The best way to get started is to follow the official guide https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/ https://kubernetes.io/docs/tutorials/stateless-application/h... You will use a tool called a “minikube” which is “mini” version of kubernetes that runs on your local machine. You should focus on to get familiar with kubectl(1) first, it’s a simple CLI tool to manage kubernetes cluster. If you have any questions & issues you can ask on Stack Overflow #kubernetes and you can also join the Slack channel.
- moistoreos 9y ago
  To follow this up, can anyone explain the benefits over Docker? I've used Docker before but am unfamiliar with Kubernetes terminology. I do understand it's an open source project by Google.
  - johnhenry 9y ago
    This isn't a full explanation, but Kubernetes isn't a direct competitor to Docker -- but rather Swarm, Docker's orchestration solution. Both work with Docker (or one of it's direct competitors such as Rkt) to orchestrate multiple containers that work in conjunction with eachother.
    - moistoreos 9y ago
      So it's essentially preference when it comes to orchestration. Thanks. :thumbs_up:
  - tapoxi 9y ago
    Kubernetes runs Docker containers (and rkt containers) but provides additional functionality like ensuring x instances of that container run across a cluster, load balancing, service discovery, and configuration management.
    - moistoreos 9y ago
      Thanks!
  - oso2k 9y ago
    If you accept the analogy that Kubernetes is an OS for the Data Center, then ‘docker’ command is the process/application launcher in that model. ‘docker’ does one thing well and Kubernetes does many things to provide a management and monitoring framework around a Container Runtime like ‘docker’ or ‘cri-o’ or the others.
    - moistoreos 9y ago
      I appreciate this analogy. Thanks for the input. +1
- oso2k 9y ago
  Full Disclosure: I work for Red Hat in the Container and PaaS Practice in Consulting. As others here have said, it’s a Container Orchestration Platform and is a largely pluggable architecture that also manages at varying extents Clustered Computing Resources, Application Resource Management, SDN Networking, DNS, Service Discovery, Load Balancing, and other concerns of “Cloud Native Application Development and Deployment“. You can try it out at http://kubernetesbyexample.com/ http://kubernetesbyexample.com/ .
  - lima 9y ago
    Red Hat's open source OpenShift PaaS[1] is amazing. It has all the missing pieces you need for a production Kubernetes cluster: documentation and playbooks on how to run a production cluster[2], a build system, a container registry with fine-grained permissions, application templates, a logging framework... Red Hat is a major contributor to Kubernetes and continually upstreams OpenShift features (like RBAC - they implemented it in OpenShift first, then upstreamed it, and then rebased OpenShift on top of it, removing the custom implementation). I'm currently looking at migrating a large enterprise setup to Kubernetes/OpenShift. [1]: https://github.com/openshift/origin https://github.com/openshift/origin [2]: https://docs.openshift.org/latest/welcome/index.html https://docs.openshift.org/latest/welcome/index.html
    - Yhippa 9y ago
      I really like OpenShift. I had a hard time grokking K8s but OpenShift is much more friendly. I was able to PoC a microservices app on OpenShift with Minishift[1]. This way you can play around with it locally and make sure you understand what you're doing. The documentation is pretty good. Red Hat has stepped up big time in this area. [1]: https://www.openshift.org/minishift/ https://www.openshift.org/minishift/
  - majewsky 9y ago
    No offense, please, but whenever I see a website describing a thing as $chain_of_buzzwords like this, I immediately close the tab. This is technically a correct definition, but completely unhelpful to a person coming in from the outside, who is not familiar with the problem space. Heck, even I am not sure what "application resource management" means, after working with Kubernetes for two years now. I could make educated guesses, but nothing certain.
- symfrog 9y ago
  The key concept to understand is that containers are just regular operating system processes with additional isolation. Most people are familiar with memory isolation between processes. The use of further isolation mechanisms for the filesystem, I/O etc. is generally termed a container [1]. Kubernetes adds an orchestration layer on top of containers to manage processes running across different operating system instances. [1] https://jvns.ca/blog/2016/10/10/what-even-is-a-container https://jvns.ca/blog/2016/10/10/what-even-is-a-container
- ShakataGaNai 9y ago
  Watch this video. It's literally K8s for kids (and quite awesomely well done): https://youtu.be/4ht22ReBjno https://youtu.be/4ht22ReBjno
- segmondy 9y ago
  Kubernetes is a container operating system. A container allows is a light weight virtual computer. Which allows you to run applications in isolation. This is useful when you might have applications with conflicting libraries. When you have multiple applications packaged in containers. You need to decide on which computer to run it, you have to deal with the networking, the load balancing if scaling it out. Upgrades and such things. Kubernetes is designed to automate all that away. You build a cluster (group of computers) You tell kubernetes you wish to deploy/run an app, kubernetes figures out where to deploy it, how to network it, link em, load balance, etc.
- _jal 9y ago
  Lots of folks have explained it well in the normal terms used for it. Here's the comparison. Compare it to init. Systemd is currently the dominant init for most Linux flavors. When your system starts and the kernel has done its thing, init takes over. It launches daemons, sets up networking, that sort of thing. After startup, it will relaunch things that crash, possibly do other things. Kubernetes is sort of an init for an emerging pattern of cross-machine systems. Like systemd, it uses configuration to figure out what should run in what order, relaunches things that fail and manages networking (albeit in a very different context), that sort of thing. There are huge differences because the problems are very different. But as a high level comparison, there are worse ones. (Especially if you're very charitable when interpreting "that sort of thing".)
  - oblio 9y ago
    I’m going to alter and copy this idea: “kubernetes is distributed init”. I’d say it’s also distributed cron and if you include the resource stuff, a distributed kernel :p
    - moosingin3space 9y ago
      Yeah, I think Kubernetes is more like a distributed OS than a distributed systemd.
- manigandham 9y ago
  You have a bunch of servers with kubernetes software installed on all of them. 1 of them is master, the rest are workers. You talk to Master through API and it talks to workers with SSH. You send the Master some YAML files that list what containers you want running, how many, and lots of optional details. Master makes it happen so it matches what you said should be running. Master also constantly checks everything so it matches what you asked for, even if something changes like a server failing. That's it.
  - brianmcc 9y ago
    Nice clear explanation. In the spirit of ELI5 I kind of think "container" is a terrible name for what seem to be super-lightweight VMs, I can't help thinking something like "instance" might have been better. In the same way in OO languages we instantiate a class to get an object instance, containerisation seems to involve creating an instance (or several instances) of an image. But I am really still learning all this stuff - I'm sure there's a reason behind it, and at the end of the day it's simply a set of terms one needs to absorb.
    - manigandham 9y ago
      Instance is probably more overloaded, but also abstract in this case. Container refers to the virtualization mechanism that is lighter than an entire operating system but still isolates several processes into their own environment. Instance of VM image => Running VM Instance of Container image => Running Container
      - brianmcc 9y ago
        That's fair enough, I guess it's just the fact that the term until very recently confused me rather than clarified things; I found it genuinely obfuscatory, as a developer with relatively light infrastructure experience. Of course I have come to terms with the fact that by now it's entrenched so we're using it, end of story - give me a few more years and I'll be pretty much adapted to it :-)
CSDude 9y ago
Good to see CSI is gaining traction. Only if we could build containers inside K8S pods safely, I would be very happy. Maintaining a stupid Docker cluster for just building containers is really a burden.
- codereflection 9y ago
  Seems someone else was able to get it that working: https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/ https://blog.jessfraz.com/post/building-container-images-sec...
  - kuschku 9y ago
    If the gitlab ci provider for kubernetes would get this feature, it’d be amazing. I could finally run gitlab’s CI safely on kubernetes and generate containers.
    - tuananh 9y ago
      i have a problem of running gitlab runner within k8s. the docker layer is not cached. is there anyway to fix this?
      - joshlambert 9y ago
        In my experience the GitLab Runner on k8s should utilized the cache. Are you using Docker-in-Docker by chance? By default, I don't think it can cache data between jobs.
        tuananh 9y ago
        can you share your config.toml for k8s runner?
cube2222 9y ago
Can anybody share their experiences with running applications that use persistent volumes on bare metal kubernetes? I mean without cloud services like Google cloud persistent disks.
- tb15 9y ago
  I've been recently experimenting with getting lustrefs usable with kubernetes, and needed some way to natively integrate it into the cluster I had. https://github.com/torchbox/k8s-hostpath-provisioner https://github.com/torchbox/k8s-hostpath-provisioner Proved to be useful, it allows you to use any mounted path on (all) a node(s) (hostpath method) to return satisfy a persistent volume claim and return a persistent volume backed by the mounted file system. A similar set up could work using bare metal, are you using something like Openstack Ironic?
  - bryanlarsen 9y ago
    For Kubernetes 1.10+ I'd recommend the local volume provisioner instead: https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume https://github.com/kubernetes-incubator/external-storage/tre...
- bryanlarsen 9y ago
  We experimented with both using cephfs via rook and glusterfs via heketi and ran into enough operational issues and speed bumps that we're just using hostPath volumes for now. IOW, they're not production ready yet.
- colek42 9y ago
  OpenEBS looks promising, but have not given it a try yet, ref: https://www.openebs.io/ https://www.openebs.io/
- pat2man 9y ago
  Redhat probably has the best production quality deployment for self hosted Kubernetes. They support running glusterfs: https://github.com/openshift/openshift-ansible/blob/master/inventory/hosts.glusterfs.native.example https://github.com/openshift/openshift-ansible/blob/master/i... Personally I wouldn't do it unless you have a RedHat contract (or already have a team that manages glusterfs), but its worth looking at.
tomerbd 9y ago
so cool, would be even cooler at 1.20
wpietri 9y ago
I'm very curious to hear field reports from people who switched to using Kubernetes in production in the last year or so. Why'd you do it? What got better and what got worse? And are you happy with the change?
- segmondy 9y ago
  At my last meetup. (Detroit Kuberentes) We heard the story of a company that was able to rebuild their entire production environment from scratch in 1 hr. They run on kuberentes, of course it's not the only story, I'm sure using something like Terraform/Cloudformation is also part of the story.
  - mrweasel 9y ago
    I've been toying around with Kubernetes (AKS on Azure) the last few weeks and have to say I rather impressed. Still being able to start from scratch and be up and running in one hour is really impressive.
    - halbritt 9y ago
      I'm using both AKS and GKE in Google and I have to say that GKE is way, way nicer to work with.
      - theflagbug 9y ago
        I'd be very interested to hear in what way you think GKE is nicer to work with, care to elaborate?
        halbritt 9y ago
        Clusters come up faster. They're easier to upgrade. Don't have to delete clusters to change them (just provision a new node pool). Persistent storage is less weird. No RBAC. Weirdness around kube-system namespace. I.e. if I create a registry there, it gets disappears with no logs or events suggesting why.
iooi 9y ago
Has anyone been using CoreDNS as Alpha in 1.9, or tried the Beta now in 1.10? What was your use case and reason for switching? How is it better than KubeDNS?
- chilledheart 9y ago
  I don't have much content on CoreDNS. But from the description in the README, https://github.com/coredns/coredns/blob/master/README.md https://github.com/coredns/coredns/blob/master/README.md, I guess it acts as a flexible and programmable entry service for underlying services and can do many applications such as Load Balance, depending on plugins.
- scurvy 9y ago
  We've been using CoreDNS for a bit now, and it's much better than KubeDNS. We found that KubeDNS would time out and drop requests from time to time. No such issues with CoreDNS. Would recommend (at least from a reliability standpoint).
  - iooi 9y ago
    Thanks for sharing. How are you monitoring timeouts and dropped requests on KubeDNS?
    - scurvy 9y ago
      We were running with shorter TTL's on service records, and upstream apps threw rashes of errors when queries timed out.
deleted 9y ago
[deleted]
humbleMouse 9y ago
Openshift from redhat is built on kubernetes. Openshift offers a free tier to try out their cloud services. I'd recommend it to anyone who wants to try it out. I'm not affiliated with redhat in any way but I have enjoyed using the openshift platform. Here's the link for anyone interested: https://www.openshift.com/pricing/index.html https://www.openshift.com/pricing/index.html
- bmaupin 9y ago
  http://openshift.io http://openshift.io is also built on Kubernetes (by virtue of using openshift.com for its deployment pieces)
sthomas1618 9y ago
Something I've been wondering about: how stable is Kubernetes service discovery? I.e. can it entirely replace something like eureka? Is there any reason not to use Kubernetes provided service discovery?
- dmourati 9y ago
  Depends if all service discovery sources and targets are all within k8s. If so, k8s works well, if not, not so much.
ascendantlogic 9y ago
They need some sort of LTS versioning. Keeping up with their breakneck development pace is a job all its own.
- manojlds 9y ago
  Kops has not released a 1.9 version yet. Even k8s projects can't keep up.
  - AlexB138 9y ago
    Kops generally stays one release behind. 1.9 is being end to end tested in the last week. I wouldn't use that as an example of not keeping up, it's the established cycle for the project.
    - justinsb 9y ago
      kops 1.9 is very close to ready now, but this is a longer lag than normal. We've historically released kops 1.x when we consider that k8s 1.x is stable, including all the networking providers and ecosystem components. That's typically about a month after release. User feedback has been that that we want to keep that, but that we should also offer an alpha/beta of 1.10 much sooner, so that users that want to try out 1.10 today can do so (and so we get feedback earlier). So watch for kops 1.10 alpha very soon, and 1.11 alpha much earlier in the 1.11 cycle.
      - iooi 9y ago
        For those that aren't aware, justinsb is the author of kops. Looking forward to the 1.9 release.
        justinsb 9y ago
        Ah - sorry, probably should have disclaimered that! I did write the original kops code, but now there's a pretty active set of contributors working on kops (and contributions are always welcome and appreciated!)
dekhn 9y ago
I switched to k8s about a year and a half ago before deploying a modest frontend/backend pair. although there is some friction, I generally like the approach (I used to use borg, so it's a pretty low barrier). The biggest problem I have is debugging all the moving parts when there are ~10+ minute async responses to config changes.
- majewsky 9y ago
  10 minutes? That sounds a lot. We usually have 1-2 minutes wait time for a changed ConfigMap to reach a running pod, or for a Deployment to roll over to a new ReplicaSet. (That's on k8s 1.4.)
FooBarWidget 9y ago
I am hearing that everybody is interested in Kubernetes, yet relatively few people are actually using Kubernetes in production. Are you using Kubernetes? And if not, what is your reason not to?
- SamLevin88 9y ago
  I'm not so sure about that last part. We (Kinvey, www.kinvey.com) have been using it for customer-facing services in production for over a year. Results have exceeded our expectations.
- majewsky 9y ago
  We are using Kubernetes since 2016 to deploy OpenStack for SAP's internal cloud. We also offer the reverse, Kubernetes on OpenStack, as-a-service to our internal customers. [1] We are all-in-all very satisfied with Kubernetes, except that it's quite messy to do right on bare metal. [1] Also available as open-source: https://github.com/sapcc/kubernikus https://github.com/sapcc/kubernikus
- dankohn1 9y ago
  Please check out the large and growing list of case studies of companies of all sizes adopting Kubernetes: https://kubernetes.io/case-studies/ https://kubernetes.io/case-studies/ Disclosure: I'm executive director of CNCF, which funds the case study write-ups.
TuringNYC 9y ago
For the folks who have implemented K8 in Production - curious if you use it to do resource management as well, or, if you also use something like Mesosphere in conjunction? Or would you stick with a like-stack (DCOS+Marathon)? There is surprisingly little online discussion/documentation on the intersect of Resource Management and Container orchestration. Not sure if it is too early in the curve, a dark art, not actually done, or what...
- ianburrell 9y ago
  Kubernetes does resource management; all cluster scheduler systems do. The difference between Kubernetes and Mesos for resource management is that resource requests and limits are optional in Kubernetes and mandatory in Mesos. It is best practice to specify resources in Kubernetes.
abledon 9y ago
I see some people who use Elixir/Erlang ecosystems then shove them into a kubernetes system. Isn't this going against what the Elixir/Erlang system already provides ? What are the usecases for this?
- Fellshard 9y ago
  I'm reading through Joe Armstrong's thesis on Erlang, and one of his main points is that concurrency programming shouldn't need to care about locality, at least in terms of correctness. Processes can run locally, or on separate machines, because each process is isolated as if it's its own VM. Containerizing it just makes for simpler deployment units, in my view. That said, there might be some pragmatic tradeoffs eventually made for the Erlang VM that I'm not familiar with yet that makes the container move more dubitable.
- troutwine 9y ago
  > Isn't this going against what the Elixir/Erlang system already provides? No? So, I guess, let's dig into that a little bit. Erlang's always kind of left the health and safety of your deployed system as a whole up to you, being preoccupied with giving you tools for understanding and maintaining its internal operation. OTP provides two semi-unique things: supervision of computation with control of failure bubble-up and hot code reloading. Hot code reloading isn't used all that much in practice, outside of domains where it's _very important_ that the whole system can never be offline and load balancing techniques are not applicable. That's a specific niche and, sure, probably one that kubernetes can't service. With regard to supervision, there's no incompatibility between OTP's approach and a deployment consisting of of ephemeral nodes that live and then die by some external mechanism. Seems to me that kubernetes is no different a deployment target in this regard than is terraform/packer, hot-swapped servers in a rack or any of the other deploy methods I've seen in my career.
noselasd 9y ago
I'm still wondering about going with plain kubernetes, or investing time in OpenShift. Any insights from people that have tried both ways ?
- kubectl 9y ago
  I deployed plain k8 to start with 2 month back and started adding more add-ons and plugins as we go. This made us be flexible abt what we use and control. So far no issues. Also our customers are inclined for k8 workload files not OpenShift.
  - dankohn1 9y ago
    Note that OpenShift is Certified Kubernetes, so you don't need to choose between OpenShift and K8s: https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/ For example, Helm charts work on OpenShift: https://blog.openshift.com/getting-started-helm-openshift/ https://blog.openshift.com/getting-started-helm-openshift/ (Disclosure: I'm the executive director of CNCF and help run the Certified Kubernetes program.)
- raesene9 9y ago
  I'm not a production user, but I've had a look at both. From what I've seen OpenShift provides a more polished and complete package than base Kubernetes, so if you're looking to get productive quick, it's probably a good option. Kubernetes base is more flexible of course, but with that flexibility comes more stuff you have to do yourself... Either way they're both based on the same tech., so experience with one largely translates across to the other
- swozey 9y ago
  I've run vanilla k8s for about 3 years now in prod but am also fairly familiar (and really like) Openshift Origin. I usually tell people asking this question the following; OO comes with a bunch of really nice quality of life improvements that are missing (but in a lot of cases can be added via 3rd party TPR/CRDs/etc) in k8s but you aren't deviating so far from k8s that you won't be able to go work on vanilla k8s in the future. Not at all. Most of the additional stuff are annotations that simply wouldn't do anything and you'd remove them if you moved from OO to k8s. I think that if you're brand new to the environment OO can really help you get running quickly. You just have to make sure that you do in fact dive in to the actual k8s yaml and deal with ingresses, prometheus, grafana, RBAC, etc at some point. I haven't used OO in awhile but I believe you could successfully do most of what I do day to day via yaml/json through the OO UI. On the flip side a lot of people will probably tell you to start from k8s, whether that's GKE or AWS or minikube or wherever and go through the k8s the hard way. Personally, and I help people quite frequently on the k8s slack, I feel like that leads a lot of people down a path of frustration. It may be perfect for your style of learning or it may just scare you off. Now when it comes to OO you are at the behest of their releases. Their most recent release was Nov 2017 and k8s 1.10 was just cut. I'm not sure what version OO is on now, evidently they changed their versioning numbers to not correspond to the k8s version. Join #kubernetes-users and #kubernetes-novice on slack.k8s.io if you need any help. It's a vibrant community. You can message me directly @ mikej if you'd like. edit: Ok OO 3.7 is k8s 1.9, that's perfect. I wait a few months before jumping into new major.
kubectl 9y ago
I am looking for, Kubectl auth login, is it available yet ?
wolfgang42 9y ago
Is there an easy way to get a single-node production-ready Kubernetes instance? I'd like to start using the Auto DevOps features that GitLab is adding, but all the tutorials I can find either have you installing minikube on your laptop or setting up a high-availability cluster with at least 3 hosts. Right now I'm using CoreOS and managing Docker containers with systemd and shell scripts, which works all right but is tedious and kind of hard to keep track of. I don't have anything that needs to autoscale or fail over or do high availability, I just want something that integrates nicely and makes it easy to deploy containers. EDIT: I should have clarified, I want to self-host this on our internal VMWare cluster, rather than run it on GKE.
- carlosdp 9y ago
  You can set the node count to 1 when you create the cluster in GitLab
  - sytse 9y ago
    For more information on how to connect the cluster to GitLab please see https://docs.gitlab.com/ee/user/project/clusters/ https://docs.gitlab.com/ee/user/project/clusters/
- MartynX 9y ago
  You can run Google Kubernetes Engine with a single node, just set the size to 1. It will complain about the high availability, etc. but it works just fine if you don't need that.
- kraig 9y ago
  i would be very cautious about calling anything single-node production-ready
  - wolfgang42 9y ago
    This is for a bunch of internal tools, so if it goes down it's more of a nuisance than anything. Is there something that makes a single-node kubernetes setup less reliable than a single server without kubernetes?
yingxie3 9y ago
Funny that this and Soloman leaving Docker showed up on the same page.
linsomniac 9y ago
Funny so many people are talking here about how fast Kubernetes is moving... That's one of the complaints about Docker Swarm that led me to rule it out...

318 points by el_duderino 9y ago

Hi..can someone ELI5 to me what Kubernetes is? Also what's the best way to get started/tutorials you can recommend for a new user? Thank you!

thinkingemote 9y ago
Kubernetes is an open source system for managing containerized applications across multiple hosts, providing basic mechanisms for deployment, maintenance, and scaling of applications
- mohaine 9y ago
  It is also mostly cloud agnostic which is nice. This gives a degree of cross cloud portability to application suites. The setup of the cluster is cloud specific but after that everything can just talk Kubernetes.
raker 9y ago
The simplest way to explain it is that it is google's version of docker swarm. Aside from that, it's a way to manage the deployment of containers across one or more hosts. Getting more traffic than usual? You have a program that detects it and sends a command to kubernetes/dockerswarm that tells it to scale up the number of web containers. Stuff like that.
zaphar 9y ago
Kubernetes orchestrates clusters of containers in a data center. It chooses where to run the containers, wires up networking between them. And keeps them running. I'm on my phone so I can't grab the link for you but a good intro is Kelsey Hightowers Kubernetes The Hard Way on GitHub.
bronak1 9y ago
Its a tool for DevOps helping orchestrate container deployments and scaling, you probably want to play around with docker before diving into to kubernetes.
peterwwillis 9y ago
Kubernetes is a thing that makes it easier for one program to talk to several other programs, by tying them together with elastic string, and putting them in whatever room it thinks they should live, and makes them as big or small as they said they needed to be. It runs in the engine bay of the truck, not in the cab of the truck, or in the container.
ams6110 9y ago
It's the latest thing that everyone is excited about and nobody will be interested in next year. Same boat as docker, openstack, mesos, et. al.
- peterwwillis 9y ago
  Most of these tools have been around for more than 5 years, many organizations have invested tons of money to use them, and each is supported financially by one or more corporations, so by that metric alone they aren't going anywhere anytime soon.
- deleted 9y ago
  [deleted]
strathos 9y ago
Katacoda has excellent tutorials/labs for Kubernetes. https://www.katacoda.com/learn https://www.katacoda.com/learn
peeyek 9y ago
The best way to get started is to follow the official guide https://kubernetes.io/docs/tutorials/stateless-application/hello-minikube/ https://kubernetes.io/docs/tutorials/stateless-application/h... You will use a tool called a “minikube” which is “mini” version of kubernetes that runs on your local machine. You should focus on to get familiar with kubectl(1) first, it’s a simple CLI tool to manage kubernetes cluster. If you have any questions & issues you can ask on Stack Overflow #kubernetes and you can also join the Slack channel.
moistoreos 9y ago
To follow this up, can anyone explain the benefits over Docker? I've used Docker before but am unfamiliar with Kubernetes terminology. I do understand it's an open source project by Google.
- johnhenry 9y ago
  This isn't a full explanation, but Kubernetes isn't a direct competitor to Docker -- but rather Swarm, Docker's orchestration solution. Both work with Docker (or one of it's direct competitors such as Rkt) to orchestrate multiple containers that work in conjunction with eachother.
  - moistoreos 9y ago
    So it's essentially preference when it comes to orchestration. Thanks. :thumbs_up:
- tapoxi 9y ago
  Kubernetes runs Docker containers (and rkt containers) but provides additional functionality like ensuring x instances of that container run across a cluster, load balancing, service discovery, and configuration management.
  - moistoreos 9y ago
    Thanks!
- oso2k 9y ago
  If you accept the analogy that Kubernetes is an OS for the Data Center, then ‘docker’ command is the process/application launcher in that model. ‘docker’ does one thing well and Kubernetes does many things to provide a management and monitoring framework around a Container Runtime like ‘docker’ or ‘cri-o’ or the others.
  - moistoreos 9y ago
    I appreciate this analogy. Thanks for the input. +1
oso2k 9y ago
Full Disclosure: I work for Red Hat in the Container and PaaS Practice in Consulting. As others here have said, it’s a Container Orchestration Platform and is a largely pluggable architecture that also manages at varying extents Clustered Computing Resources, Application Resource Management, SDN Networking, DNS, Service Discovery, Load Balancing, and other concerns of “Cloud Native Application Development and Deployment“. You can try it out at http://kubernetesbyexample.com/ http://kubernetesbyexample.com/ .
- lima 9y ago
  Red Hat's open source OpenShift PaaS[1] is amazing. It has all the missing pieces you need for a production Kubernetes cluster: documentation and playbooks on how to run a production cluster[2], a build system, a container registry with fine-grained permissions, application templates, a logging framework... Red Hat is a major contributor to Kubernetes and continually upstreams OpenShift features (like RBAC - they implemented it in OpenShift first, then upstreamed it, and then rebased OpenShift on top of it, removing the custom implementation). I'm currently looking at migrating a large enterprise setup to Kubernetes/OpenShift. [1]: https://github.com/openshift/origin https://github.com/openshift/origin [2]: https://docs.openshift.org/latest/welcome/index.html https://docs.openshift.org/latest/welcome/index.html
  - Yhippa 9y ago
    I really like OpenShift. I had a hard time grokking K8s but OpenShift is much more friendly. I was able to PoC a microservices app on OpenShift with Minishift[1]. This way you can play around with it locally and make sure you understand what you're doing. The documentation is pretty good. Red Hat has stepped up big time in this area. [1]: https://www.openshift.org/minishift/ https://www.openshift.org/minishift/
- majewsky 9y ago
  No offense, please, but whenever I see a website describing a thing as $chain_of_buzzwords like this, I immediately close the tab. This is technically a correct definition, but completely unhelpful to a person coming in from the outside, who is not familiar with the problem space. Heck, even I am not sure what "application resource management" means, after working with Kubernetes for two years now. I could make educated guesses, but nothing certain.
symfrog 9y ago
The key concept to understand is that containers are just regular operating system processes with additional isolation. Most people are familiar with memory isolation between processes. The use of further isolation mechanisms for the filesystem, I/O etc. is generally termed a container [1]. Kubernetes adds an orchestration layer on top of containers to manage processes running across different operating system instances. [1] https://jvns.ca/blog/2016/10/10/what-even-is-a-container https://jvns.ca/blog/2016/10/10/what-even-is-a-container
ShakataGaNai 9y ago
Watch this video. It's literally K8s for kids (and quite awesomely well done): https://youtu.be/4ht22ReBjno https://youtu.be/4ht22ReBjno
segmondy 9y ago
Kubernetes is a container operating system. A container allows is a light weight virtual computer. Which allows you to run applications in isolation. This is useful when you might have applications with conflicting libraries. When you have multiple applications packaged in containers. You need to decide on which computer to run it, you have to deal with the networking, the load balancing if scaling it out. Upgrades and such things. Kubernetes is designed to automate all that away. You build a cluster (group of computers) You tell kubernetes you wish to deploy/run an app, kubernetes figures out where to deploy it, how to network it, link em, load balance, etc.
_jal 9y ago
Lots of folks have explained it well in the normal terms used for it. Here's the comparison. Compare it to init. Systemd is currently the dominant init for most Linux flavors. When your system starts and the kernel has done its thing, init takes over. It launches daemons, sets up networking, that sort of thing. After startup, it will relaunch things that crash, possibly do other things. Kubernetes is sort of an init for an emerging pattern of cross-machine systems. Like systemd, it uses configuration to figure out what should run in what order, relaunches things that fail and manages networking (albeit in a very different context), that sort of thing. There are huge differences because the problems are very different. But as a high level comparison, there are worse ones. (Especially if you're very charitable when interpreting "that sort of thing".)
- oblio 9y ago
  I’m going to alter and copy this idea: “kubernetes is distributed init”. I’d say it’s also distributed cron and if you include the resource stuff, a distributed kernel :p
  - moosingin3space 9y ago
    Yeah, I think Kubernetes is more like a distributed OS than a distributed systemd.
manigandham 9y ago
You have a bunch of servers with kubernetes software installed on all of them. 1 of them is master, the rest are workers. You talk to Master through API and it talks to workers with SSH. You send the Master some YAML files that list what containers you want running, how many, and lots of optional details. Master makes it happen so it matches what you said should be running. Master also constantly checks everything so it matches what you asked for, even if something changes like a server failing. That's it.
- brianmcc 9y ago
  Nice clear explanation. In the spirit of ELI5 I kind of think "container" is a terrible name for what seem to be super-lightweight VMs, I can't help thinking something like "instance" might have been better. In the same way in OO languages we instantiate a class to get an object instance, containerisation seems to involve creating an instance (or several instances) of an image. But I am really still learning all this stuff - I'm sure there's a reason behind it, and at the end of the day it's simply a set of terms one needs to absorb.
  - manigandham 9y ago
    Instance is probably more overloaded, but also abstract in this case. Container refers to the virtualization mechanism that is lighter than an entire operating system but still isolates several processes into their own environment. Instance of VM image => Running VM Instance of Container image => Running Container
    - brianmcc 9y ago
      That's fair enough, I guess it's just the fact that the term until very recently confused me rather than clarified things; I found it genuinely obfuscatory, as a developer with relatively light infrastructure experience. Of course I have come to terms with the fact that by now it's entrenched so we're using it, end of story - give me a few more years and I'll be pretty much adapted to it :-)

CSDude 9y ago

Good to see CSI is gaining traction. Only if we could build containers inside K8S pods safely, I would be very happy. Maintaining a stupid Docker cluster for just building containers is really a burden.

codereflection 9y ago
Seems someone else was able to get it that working: https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/ https://blog.jessfraz.com/post/building-container-images-sec...
- kuschku 9y ago
  If the gitlab ci provider for kubernetes would get this feature, it’d be amazing. I could finally run gitlab’s CI safely on kubernetes and generate containers.
  - tuananh 9y ago
    i have a problem of running gitlab runner within k8s. the docker layer is not cached. is there anyway to fix this?
    - joshlambert 9y ago
      In my experience the GitLab Runner on k8s should utilized the cache. Are you using Docker-in-Docker by chance? By default, I don't think it can cache data between jobs.
      - tuananh 9y ago
        can you share your config.toml for k8s runner?

cube2222 9y ago

Can anybody share their experiences with running applications that use persistent volumes on bare metal kubernetes? I mean without cloud services like Google cloud persistent disks.

tb15 9y ago
I've been recently experimenting with getting lustrefs usable with kubernetes, and needed some way to natively integrate it into the cluster I had. https://github.com/torchbox/k8s-hostpath-provisioner https://github.com/torchbox/k8s-hostpath-provisioner Proved to be useful, it allows you to use any mounted path on (all) a node(s) (hostpath method) to return satisfy a persistent volume claim and return a persistent volume backed by the mounted file system. A similar set up could work using bare metal, are you using something like Openstack Ironic?
- bryanlarsen 9y ago
  For Kubernetes 1.10+ I'd recommend the local volume provisioner instead: https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume https://github.com/kubernetes-incubator/external-storage/tre...
bryanlarsen 9y ago
We experimented with both using cephfs via rook and glusterfs via heketi and ran into enough operational issues and speed bumps that we're just using hostPath volumes for now. IOW, they're not production ready yet.
colek42 9y ago
OpenEBS looks promising, but have not given it a try yet, ref: https://www.openebs.io/ https://www.openebs.io/
pat2man 9y ago
Redhat probably has the best production quality deployment for self hosted Kubernetes. They support running glusterfs: https://github.com/openshift/openshift-ansible/blob/master/inventory/hosts.glusterfs.native.example https://github.com/openshift/openshift-ansible/blob/master/i... Personally I wouldn't do it unless you have a RedHat contract (or already have a team that manages glusterfs), but its worth looking at.

tomerbd 9y ago

so cool, would be even cooler at 1.20

wpietri 9y ago

I'm very curious to hear field reports from people who switched to using Kubernetes in production in the last year or so. Why'd you do it? What got better and what got worse? And are you happy with the change?

segmondy 9y ago
At my last meetup. (Detroit Kuberentes) We heard the story of a company that was able to rebuild their entire production environment from scratch in 1 hr. They run on kuberentes, of course it's not the only story, I'm sure using something like Terraform/Cloudformation is also part of the story.
- mrweasel 9y ago
  I've been toying around with Kubernetes (AKS on Azure) the last few weeks and have to say I rather impressed. Still being able to start from scratch and be up and running in one hour is really impressive.
  - halbritt 9y ago
    I'm using both AKS and GKE in Google and I have to say that GKE is way, way nicer to work with.
    - theflagbug 9y ago
      I'd be very interested to hear in what way you think GKE is nicer to work with, care to elaborate?
      - halbritt 9y ago
        Clusters come up faster. They're easier to upgrade. Don't have to delete clusters to change them (just provision a new node pool). Persistent storage is less weird. No RBAC. Weirdness around kube-system namespace. I.e. if I create a registry there, it gets disappears with no logs or events suggesting why.

iooi 9y ago

Has anyone been using CoreDNS as Alpha in 1.9, or tried the Beta now in 1.10? What was your use case and reason for switching? How is it better than KubeDNS?

chilledheart 9y ago
I don't have much content on CoreDNS. But from the description in the README, https://github.com/coredns/coredns/blob/master/README.md https://github.com/coredns/coredns/blob/master/README.md, I guess it acts as a flexible and programmable entry service for underlying services and can do many applications such as Load Balance, depending on plugins.
scurvy 9y ago
We've been using CoreDNS for a bit now, and it's much better than KubeDNS. We found that KubeDNS would time out and drop requests from time to time. No such issues with CoreDNS. Would recommend (at least from a reliability standpoint).
- iooi 9y ago
  Thanks for sharing. How are you monitoring timeouts and dropped requests on KubeDNS?
  - scurvy 9y ago
    We were running with shorter TTL's on service records, and upstream apps threw rashes of errors when queries timed out.

deleted 9y ago

[deleted]

humbleMouse 9y ago

Openshift from redhat is built on kubernetes. Openshift offers a free tier to try out their cloud services. I'd recommend it to anyone who wants to try it out. I'm not affiliated with redhat in any way but I have enjoyed using the openshift platform. Here's the link for anyone interested: https://www.openshift.com/pricing/index.html https://www.openshift.com/pricing/index.html

bmaupin 9y ago
http://openshift.io http://openshift.io is also built on Kubernetes (by virtue of using openshift.com for its deployment pieces)

sthomas1618 9y ago

Something I've been wondering about: how stable is Kubernetes service discovery? I.e. can it entirely replace something like eureka? Is there any reason not to use Kubernetes provided service discovery?

dmourati 9y ago
Depends if all service discovery sources and targets are all within k8s. If so, k8s works well, if not, not so much.

ascendantlogic 9y ago

They need some sort of LTS versioning. Keeping up with their breakneck development pace is a job all its own.

manojlds 9y ago
Kops has not released a 1.9 version yet. Even k8s projects can't keep up.
- AlexB138 9y ago
  Kops generally stays one release behind. 1.9 is being end to end tested in the last week. I wouldn't use that as an example of not keeping up, it's the established cycle for the project.
  - justinsb 9y ago
    kops 1.9 is very close to ready now, but this is a longer lag than normal. We've historically released kops 1.x when we consider that k8s 1.x is stable, including all the networking providers and ecosystem components. That's typically about a month after release. User feedback has been that that we want to keep that, but that we should also offer an alpha/beta of 1.10 much sooner, so that users that want to try out 1.10 today can do so (and so we get feedback earlier). So watch for kops 1.10 alpha very soon, and 1.11 alpha much earlier in the 1.11 cycle.
    - iooi 9y ago
      For those that aren't aware, justinsb is the author of kops. Looking forward to the 1.9 release.
      - justinsb 9y ago
        Ah - sorry, probably should have disclaimered that! I did write the original kops code, but now there's a pretty active set of contributors working on kops (and contributions are always welcome and appreciated!)

dekhn 9y ago

I switched to k8s about a year and a half ago before deploying a modest frontend/backend pair. although there is some friction, I generally like the approach (I used to use borg, so it's a pretty low barrier). The biggest problem I have is debugging all the moving parts when there are ~10+ minute async responses to config changes.

majewsky 9y ago
10 minutes? That sounds a lot. We usually have 1-2 minutes wait time for a changed ConfigMap to reach a running pod, or for a Deployment to roll over to a new ReplicaSet. (That's on k8s 1.4.)

FooBarWidget 9y ago

I am hearing that everybody is interested in Kubernetes, yet relatively few people are actually using Kubernetes in production. Are you using Kubernetes? And if not, what is your reason not to?

SamLevin88 9y ago
I'm not so sure about that last part. We (Kinvey, www.kinvey.com) have been using it for customer-facing services in production for over a year. Results have exceeded our expectations.
majewsky 9y ago
We are using Kubernetes since 2016 to deploy OpenStack for SAP's internal cloud. We also offer the reverse, Kubernetes on OpenStack, as-a-service to our internal customers. [1] We are all-in-all very satisfied with Kubernetes, except that it's quite messy to do right on bare metal. [1] Also available as open-source: https://github.com/sapcc/kubernikus https://github.com/sapcc/kubernikus
dankohn1 9y ago
Please check out the large and growing list of case studies of companies of all sizes adopting Kubernetes: https://kubernetes.io/case-studies/ https://kubernetes.io/case-studies/ Disclosure: I'm executive director of CNCF, which funds the case study write-ups.

TuringNYC 9y ago

For the folks who have implemented K8 in Production - curious if you use it to do resource management as well, or, if you also use something like Mesosphere in conjunction? Or would you stick with a like-stack (DCOS+Marathon)? There is surprisingly little online discussion/documentation on the intersect of Resource Management and Container orchestration. Not sure if it is too early in the curve, a dark art, not actually done, or what...

ianburrell 9y ago
Kubernetes does resource management; all cluster scheduler systems do. The difference between Kubernetes and Mesos for resource management is that resource requests and limits are optional in Kubernetes and mandatory in Mesos. It is best practice to specify resources in Kubernetes.

abledon 9y ago

I see some people who use Elixir/Erlang ecosystems then shove them into a kubernetes system. Isn't this going against what the Elixir/Erlang system already provides ? What are the usecases for this?

Fellshard 9y ago
I'm reading through Joe Armstrong's thesis on Erlang, and one of his main points is that concurrency programming shouldn't need to care about locality, at least in terms of correctness. Processes can run locally, or on separate machines, because each process is isolated as if it's its own VM. Containerizing it just makes for simpler deployment units, in my view. That said, there might be some pragmatic tradeoffs eventually made for the Erlang VM that I'm not familiar with yet that makes the container move more dubitable.
troutwine 9y ago
> Isn't this going against what the Elixir/Erlang system already provides? No? So, I guess, let's dig into that a little bit. Erlang's always kind of left the health and safety of your deployed system as a whole up to you, being preoccupied with giving you tools for understanding and maintaining its internal operation. OTP provides two semi-unique things: supervision of computation with control of failure bubble-up and hot code reloading. Hot code reloading isn't used all that much in practice, outside of domains where it's _very important_ that the whole system can never be offline and load balancing techniques are not applicable. That's a specific niche and, sure, probably one that kubernetes can't service. With regard to supervision, there's no incompatibility between OTP's approach and a deployment consisting of of ephemeral nodes that live and then die by some external mechanism. Seems to me that kubernetes is no different a deployment target in this regard than is terraform/packer, hot-swapped servers in a rack or any of the other deploy methods I've seen in my career.

noselasd 9y ago

I'm still wondering about going with plain kubernetes, or investing time in OpenShift. Any insights from people that have tried both ways ?

kubectl 9y ago
I deployed plain k8 to start with 2 month back and started adding more add-ons and plugins as we go. This made us be flexible abt what we use and control. So far no issues. Also our customers are inclined for k8 workload files not OpenShift.
- dankohn1 9y ago
  Note that OpenShift is Certified Kubernetes, so you don't need to choose between OpenShift and K8s: https://www.cncf.io/certification/software-conformance/ https://www.cncf.io/certification/software-conformance/ For example, Helm charts work on OpenShift: https://blog.openshift.com/getting-started-helm-openshift/ https://blog.openshift.com/getting-started-helm-openshift/ (Disclosure: I'm the executive director of CNCF and help run the Certified Kubernetes program.)
raesene9 9y ago
I'm not a production user, but I've had a look at both. From what I've seen OpenShift provides a more polished and complete package than base Kubernetes, so if you're looking to get productive quick, it's probably a good option. Kubernetes base is more flexible of course, but with that flexibility comes more stuff you have to do yourself... Either way they're both based on the same tech., so experience with one largely translates across to the other
swozey 9y ago
I've run vanilla k8s for about 3 years now in prod but am also fairly familiar (and really like) Openshift Origin. I usually tell people asking this question the following; OO comes with a bunch of really nice quality of life improvements that are missing (but in a lot of cases can be added via 3rd party TPR/CRDs/etc) in k8s but you aren't deviating so far from k8s that you won't be able to go work on vanilla k8s in the future. Not at all. Most of the additional stuff are annotations that simply wouldn't do anything and you'd remove them if you moved from OO to k8s. I think that if you're brand new to the environment OO can really help you get running quickly. You just have to make sure that you do in fact dive in to the actual k8s yaml and deal with ingresses, prometheus, grafana, RBAC, etc at some point. I haven't used OO in awhile but I believe you could successfully do most of what I do day to day via yaml/json through the OO UI. On the flip side a lot of people will probably tell you to start from k8s, whether that's GKE or AWS or minikube or wherever and go through the k8s the hard way. Personally, and I help people quite frequently on the k8s slack, I feel like that leads a lot of people down a path of frustration. It may be perfect for your style of learning or it may just scare you off. Now when it comes to OO you are at the behest of their releases. Their most recent release was Nov 2017 and k8s 1.10 was just cut. I'm not sure what version OO is on now, evidently they changed their versioning numbers to not correspond to the k8s version. Join #kubernetes-users and #kubernetes-novice on slack.k8s.io if you need any help. It's a vibrant community. You can message me directly @ mikej if you'd like. edit: Ok OO 3.7 is k8s 1.9, that's perfect. I wait a few months before jumping into new major.

kubectl 9y ago

I am looking for, Kubectl auth login, is it available yet ?

wolfgang42 9y ago

Is there an easy way to get a single-node production-ready Kubernetes instance? I'd like to start using the Auto DevOps features that GitLab is adding, but all the tutorials I can find either have you installing minikube on your laptop or setting up a high-availability cluster with at least 3 hosts. Right now I'm using CoreOS and managing Docker containers with systemd and shell scripts, which works all right but is tedious and kind of hard to keep track of. I don't have anything that needs to autoscale or fail over or do high availability, I just want something that integrates nicely and makes it easy to deploy containers. EDIT: I should have clarified, I want to self-host this on our internal VMWare cluster, rather than run it on GKE.

carlosdp 9y ago
You can set the node count to 1 when you create the cluster in GitLab
- sytse 9y ago
  For more information on how to connect the cluster to GitLab please see https://docs.gitlab.com/ee/user/project/clusters/ https://docs.gitlab.com/ee/user/project/clusters/
MartynX 9y ago
You can run Google Kubernetes Engine with a single node, just set the size to 1. It will complain about the high availability, etc. but it works just fine if you don't need that.
kraig 9y ago
i would be very cautious about calling anything single-node production-ready
- wolfgang42 9y ago
  This is for a bunch of internal tools, so if it goes down it's more of a nuisance than anything. Is there something that makes a single-node kubernetes setup less reliable than a single server without kubernetes?

yingxie3 9y ago

Funny that this and Soloman leaving Docker showed up on the same page.

linsomniac 9y ago

Funny so many people are talking here about how fast Kubernetes is moving... That's one of the complaints about Docker Swarm that led me to rule it out...