9 ms·

A Tutorial Introduction to Kubernetes

gingerlime 8y ago
EDIT: I should preface my little rant by saying that this post is one of the best I've seen at explaining the basic concepts of Kubernetes. But obviously I'm not an expert :) > The Kubernetes API should now be available at http://localhost:8001 http://localhost:8001, and the dashboard at this rather complicated URL. It used to be reachable at http://localhost:8001/ui http://localhost:8001/ui, but this has been changed due to what I gather are security reasons. I was playing around with GCE Hosted Kubernetes about a year ago, and things were pretty clear as far as I recall. I've read lots of positive things, and figured it's a good way to start. Then I tried again recently, and I couldn't even get to the dashboard. Eventually after several cryptic StackOverflow copy&pastes I managed to load it (don't even remember how), only for the session to expire after 10 minutes or so... It was utterly frustrating. I didn't actually get to the more interesting part I was planning to play with as a result... People say that there's a learning curve, and I get it. And also I'm not even trying to install Kubernetes on my own, but try to use a hosted service. I'm also pretty switched on when it comes to security and trying new things (or I'd like to think I am), but there are some things that feel like too much of an obstacle for me unfortunately.
- lnsp 8y ago
  GKE actively disadvises from using the Kubernetes Dashboard and recommends their dashboard as a replacement. On their website [1] they list the following: > Caution: As of September 2017, Kubernetes Dashboard is deprecated in Kubernetes Engine. You can use the dashboards described on this page to monitor your clusters' performance, workloads, and resources. [1] https://cloud.google.com/kubernetes-engine/docs/concepts/dashboards https://cloud.google.com/kubernetes-engine/docs/concepts/das...
  - gingerlime 8y ago
    That's good to know. I guess I was just heading in the wrong direction from the start. I think I did try the built-in dashboard on the GCE web console, but somehow didn't see any of the stuff I was expecting (but maybe I've just assumed the "real" dashboard is the one I was trying to load).
    - yebyen 8y ago
      I was looking for Heapster and InfluxDB but I think I was supposed to find Prometheus... This stuff moves very fast. You can be excused for blinking, IMHO!
mikehollinger 8y ago
We use kubernetes to spin up the application that I work on (in private cloud and at some point in hybrid and public cloud) deployments. It’s an end user installed tool. In deployment, about 1/4 of the new installations fail because of some problem or another. Either the GPU plugins for NVIDIA weren’t loaded correctly, kube-dns won’t start because docker0 isn’t in a “trusted” zone in redhat (not being in trusted seems to cause iptables to subtly screw up container to container communication between the various private networks), or helm just decides that it can’t start. Are we doing it wrong? We’re using hyperkube and k8s 1.8 which came out around q4 of last year. Almost all of these I can trace back to user error (ie we told folks to do X, they didn’t, and stuff broke). We’re now having to write a preflight checklist of sorts that the app runs through to make sure A bunch of stuff is “ok.” That in itself becomes brittle in my experience so I’m reluctant to do that.
- afroisalreadyin 8y ago
  What is now happening around Kubernetes is that various companies and projects are coming up with "distributions" of their own. These package the underlying OS, the Kubernetes binaries, and a number of built-in system pods that take care of various things such as logging, scheduling and monitoring. Red Hat's OpenShift is one of them, as well Weaveworks. Maybe try having a look at one such solution?
  - psynapse 8y ago
    There is also the Canonical Distribution of Kubernetes (CDK). If you want to try a stripped down version on your local machine, kubernetes-core can be installed into LXD containers via conjure-up with a click of the fingers - nice for playing around.
- wikibob 8y ago
  It requires considerable operational experience and effort to run well on bare metal. Have you considered experimenting with a managed Kubernetes offering? Out of EKS, AKS, and GKE, I cannot recommend GKE highly enough. Even if you can’t use it for production, it’s highly worth setting up a prototype environment on GKE to see what it gives you. I believe they now have GPU support, including support for preemptible GPUs (much cheaper). Also GKE does a good job of staying up to date with releases. They are on 1.10, with support expected soon for 1.11. They _fully_ manage the upgrade of both the Kubernetes and etcd masters, and the worker nodes. As mentioned in the linked blog post, Kubernetes is a fast moving project, and to use it you should plan and allocate significant resources in your team to keeping up with the new releases. There are a large number of fixes and improvements since 1.8 and I would look very seriously at both upgrading, and changing your processes to allow you to stay closer to the current release version. The Kubernetes project does not, and has no current plans to, have a long term support release.
brianmcc 8y ago
Been looking for a decent intro write up for some time - this could be it!
auslander 8y ago
If you are on AWS, try ECS first, it is much simpler and has main features you need: HA, autoscaling, version control. When you'll mature from using Docker to high volume production, you should ditch containers at all, they are good for prototyping and testing, but not for production loads and production security.
- wikibob 8y ago
  ECS is simple to get started, but does not have a vast array of features that are necessary to run a service reliably at scale. It is possible to add many of these to ECS, but it’s left as an exercise to the reader to write up various API’s, cloudwatch, Lambda, etc. I can provide more specific details if there’s questions.
  - auslander 8y ago
    Thank you. Yes, could you name the features ECS is missing for running at scale reliably, please?
    - shawabawa3 8y ago
      I've recently moved from ECS to Kubernetes, and much prefer kubernetes In general ECS is fine when you have 1 or 2 services, but once you have 10s or 100s it gets pretty unusable. Logging: While ECS ships everything to cloudwatch, actually accessing them is a nightmare and usually more effort than it's worth. It's very easy on k8s to get logs shipped from every container to elasticsearch and browse with kibana. Autoscaling: Possibly less of an issue now that fargate's a thing - but fargate's expensive. Kubernetes makes it easy to set up both node level autoscaling, and deployment level autoscaling Ingress: With ECS you have to set up loadbalancers for every service individually - in k8s once the controller's properly set up every service defines its own ingress, and the only manual change you have to do is create a DNS entry (only if you can't use a wildcard) Metrics: Prometheus is very ingrained in kubernetes, and you get a wealth of information from every service almost "for free" once you've installed it. And more: service meshes, secret control (through e.g. vault), declarative definitions, provider agnostic Kubernetes has a huge startup cost and learning curve, but is incredibly powerful once you're up and running
colonelpopcorn 8y ago
I really want to like Kubernetes, but going beyond the basics seems to require a way higher understanding of systems engineering than I currently have. Yes I know you can create container networks and stateful pods with attached storage, but how is always seemingly beyond me. Network and storage in distributed computing is hard and Kubernetes seems to be a slightly more magical bullet than Docker Swarm alone.
- kgoutham93 8y ago
  Completely agree... Going beyond the basics is really hard. But as I understand it's because of inadequate knowledge of advanced network/storage virtualization concepts. Any help on how to get started in those? As a side note I have decent knowledge of basic networking/storage.
- outworlder 8y ago
  Look up persistent volume claims and dynamic provisioning. If you cluster is properly configured, then you are one yaml file away from having that. By "configured properly" this means having the correct cloud provider set, otherwise it cannot talk to the APIs. To be fair, this stateful containers in general are a relatively new thing in K8s and support has been improving. Also, K8s is trying to do and abstract away a lot, it is more like a distributed operating system by itself. So it is more complicated than swarm.
mosselman 8y ago
What I find the hardest to figure out is how to properly deploy databases in kubernetes. What kind of volumes should I use and how do I configure them for production instead of some hello world situation?
- djsumdog 8y ago
  I think in ever docker ecosystem shop I've worked at, Databases are always on RDS. For large storage, we bought NAS solutions in the local data center. Our DC/OS cluster was primarily for things that didn't need to keep any sort of state. We had a few containers that needed NFS access and that was a lot of special work with the platform team, using labels to pin containers to certain nodes.
- afroisalreadyin 8y ago
  A database would normally be deployed as a stateful set, with the data stored in a volume, created and maintained by a PVC (persistent volume claim). This mechanism ensures that in the case of your pod dying and getting recreated on a different volume, it would see the same data as before, with the PVC getting mounted on the same pod. This mechanism is still one of the weaker links in the Kubernetes chain, in my experience, as the PVC feature is implemented completely differently on each platform, and is prone to bugs (see https://github.com/kubernetes/kubernetes/issues/60101 https://github.com/kubernetes/kubernetes/issues/60101, from Kubernetes version 1.7 on Azure, for example). Kelsey Hightower himself mentioned in a tweet that he prefers hosted solutions for persistency, and Kubernetes for stateless applications. Also, you should have a look at operators for managing applications on Kubernetes, such as this one for PostgreSQL: https://github.com/zalando-incubator/postgres-operator https://github.com/zalando-incubator/postgres-operator.
- threeseed 8y ago
  Depends on your database. Some like Cassandra are easier since it is multi-master and so you could just use local storage and if the node goes down then it's not the end of the world. There is an interesting solution for MySQL: https://vitess.io https://vitess.io which apparently is used by Youtube and Slack.
djsumdog 8y ago
I gave up on my own Kubernetes writeup a while back. I just had a lot of trouble with basic networking configuration, logging, etc. I've been at one shop with a large scale DC/OS installation. You can run a k8s scheduler on DC/OS, but by default it uses Marathon. DC/OS has it's own problems for sure, and both tools require a full time team of at least 3 people (we had 8~10) and there are a lot of things that will probably need to be customized for your shop (which labels to use, scripts to setup your ingress/egress points in AWS, HAproxy configuration or marathon-lb configuration .. which is just a haproxy container/wrapper), but I think I still prefer marathon. I briefly played with nomad and which I had spent more time with it. I know people from at least one startup around where I live using it in production. It seems to be a bit more minimal and potentially more sane. The thing I hate about all of these is there is no 1 to n scaling. For a simple project, I can't just setup one node with a minimal scheduler. DC/OS is going to cost you ~$120 a month for one non-redundant node: https://penguindreams.org/blog/installing-mesosphere-dcos-on-small-digital-ocean-droplets/ https://penguindreams.org/blog/installing-mesosphere-dcos-on... I hear people talk about minicube, but that's not something you can expand from one node to 100 right? You still have to build out a real k8s cluster at some point. All of these tools are just frontends around a scheduling and container engine (typically Docker and VMs) that track which containers are running where and track networking between nodes (and you often still have to chose and configure that networking layer .. weavenet, flannel, etc). I know someone will probably mention Rancer, and I should probably look at it again, but last time I looked I felt it was all point-n-click GUI and not enough command line flags (or at least not enough documented CLI) to really be used in an infrastructure as code fashion. I feel like there's still a big missing piece of the docker ecosystem, a really simple scheduler that can easily be stood up on new nodes and attach them to an existing cluster, and has a simply way of handling public IPs for web apps/haproxy containers. I know you can do this with K8s, DC/OS, etc. But there is a lot of prep work that has to be done first.
- colonelpopcorn 8y ago
  I have yet to see a good tutorial that shows an automated build of a kubernetes cluster. Yeah, you can use GKE, but that gets to be prohibitively expensive.
tobbyb 8y ago
We are working on a project with standard LXC containers [1] which tries to make orchestration and some of this stuff especially networking simpler. We support provisioning servers, building overlay networks with Vxlan, BGP & Wireguard, distributed storage and rolling out things like service discovery, load balancers and HA. It may be worth exploring for those struggling with some of the complexity around container deployments. At the minimum it will help you understand more about containers, networking and orchestration. [1] https://www.flockport.com https://www.flockport.com
rcconf 8y ago
I've been using Kubernetes on Azure and GCE recently and it's absolutely wonderful. I was able to setup an entire ecosystem from scratch in a week that scales well and can be managed in one location. When I first looked at Kubernetes, the complicated part was setting it up on a cluster. If you use Kubernetes on GCE, or Azure, you don't have to do that step, everything else is ready to go for you! - Automatic scaling of your application - Service discovery - Secrets and config management - Logging in one central dashboard - Able to deploy various complicated, distributed, pieces of software very easily using Helm (Jenkins, Kafka, Grafana + Prometheus) - Able to add new nodes to the cluster easily - Health checks and automatic restarts - Able to deploy any container to your cluster in a really simple way (if you look at Deployments, it's really simple.) - Switch between cloud providers and still maintain the same workflow. I won't ever touch Ansible again, I really prefer the Kubernetes way of handling operations (it's like a live organism instead of something you apply changes to.) Also, the entire argument that you probably don't need Kubernetes because your organization doesn't have 10s, or 100s of nodes just doesn't make sense after using it. Having a Kubernetes cluster with 3 nodes is 100% worth it, even for rather simple applications in my opinion. The benefits are just way too good.
- ramoz 8y ago
  How was your networking experience? Any exp running within Enterprise VPN? Private image repos in the network? And if you needed to leave GKE to another provider or packaged , how would that look? Thanks for any answers!
- TuringNYC 8y ago
  I'm glad you mentioned Azure and GCE -- I started off with minikube locally and things didnt really become magical until i started using a kops-deployed AWS-EC2 cluster and saw the magic of elastic scaling, ease of deployment, and migrate-ability. PS - Could you share which books/websites/resources you used to get up to speed to the point where you're at now?
  - chadmeister 8y ago
    Would you mind sharing links to any of the resources you used to get yourself going with kops on AWS? I'm in the midst of setting up my first cluster as we speak.
merinowool 8y ago
I wish I could find a tutorial for bare metal: - how to setup cert creation for fqdn with Cloudflare - storage - ingress to support multiple ips glued to different nodes (so if service gets IP x it gets routed through node z that has this external IP) I spent 6 months trying to do that and no luck.
joshschreuder 8y ago
Scott McCloud, author of the great book Understanding Comics, also did a comic book introduction to Kubernetes: https://cloud.google.com/kubernetes-engine/kubernetes-comic/ https://cloud.google.com/kubernetes-engine/kubernetes-comic/
- deleted 8y ago
  [deleted]
- emmelaich 8y ago
  ..including interactive demo start! Nice. PS. McCloud? I sense nominative determinism at play.
ohiovr 8y ago
What is the typical use case for clustered applications? What size organization really needs it? I understand that a simple nginx static site can accept thousands of simultaneous connections per host. That sounds pretty huge to me. If you were to sell kubernetes based solutions who would you consider selling to? What makes kubernetes fundamentally superior to docker swarm?

EDIT: I should preface my little rant by saying that this post is one of the best I've seen at explaining the basic concepts of Kubernetes. But obviously I'm not an expert :) > The Kubernetes API should now be available at http://localhost:8001 http://localhost:8001, and the dashboard at this rather complicated URL. It used to be reachable at http://localhost:8001/ui http://localhost:8001/ui, but this has been changed due to what I gather are security reasons. I was playing around with GCE Hosted Kubernetes about a year ago, and things were pretty clear as far as I recall. I've read lots of positive things, and figured it's a good way to start. Then I tried again recently, and I couldn't even get to the dashboard. Eventually after several cryptic StackOverflow copy&pastes I managed to load it (don't even remember how), only for the session to expire after 10 minutes or so... It was utterly frustrating. I didn't actually get to the more interesting part I was planning to play with as a result... People say that there's a learning curve, and I get it. And also I'm not even trying to install Kubernetes on my own, but try to use a hosted service. I'm also pretty switched on when it comes to security and trying new things (or I'd like to think I am), but there are some things that feel like too much of an obstacle for me unfortunately.

lnsp 8y ago
GKE actively disadvises from using the Kubernetes Dashboard and recommends their dashboard as a replacement. On their website [1] they list the following: > Caution: As of September 2017, Kubernetes Dashboard is deprecated in Kubernetes Engine. You can use the dashboards described on this page to monitor your clusters' performance, workloads, and resources. [1] https://cloud.google.com/kubernetes-engine/docs/concepts/dashboards https://cloud.google.com/kubernetes-engine/docs/concepts/das...
- gingerlime 8y ago
  That's good to know. I guess I was just heading in the wrong direction from the start. I think I did try the built-in dashboard on the GCE web console, but somehow didn't see any of the stuff I was expecting (but maybe I've just assumed the "real" dashboard is the one I was trying to load).
  - yebyen 8y ago
    I was looking for Heapster and InfluxDB but I think I was supposed to find Prometheus... This stuff moves very fast. You can be excused for blinking, IMHO!

mikehollinger 8y ago

We use kubernetes to spin up the application that I work on (in private cloud and at some point in hybrid and public cloud) deployments. It’s an end user installed tool. In deployment, about 1/4 of the new installations fail because of some problem or another. Either the GPU plugins for NVIDIA weren’t loaded correctly, kube-dns won’t start because docker0 isn’t in a “trusted” zone in redhat (not being in trusted seems to cause iptables to subtly screw up container to container communication between the various private networks), or helm just decides that it can’t start. Are we doing it wrong? We’re using hyperkube and k8s 1.8 which came out around q4 of last year. Almost all of these I can trace back to user error (ie we told folks to do X, they didn’t, and stuff broke). We’re now having to write a preflight checklist of sorts that the app runs through to make sure A bunch of stuff is “ok.” That in itself becomes brittle in my experience so I’m reluctant to do that.

afroisalreadyin 8y ago
What is now happening around Kubernetes is that various companies and projects are coming up with "distributions" of their own. These package the underlying OS, the Kubernetes binaries, and a number of built-in system pods that take care of various things such as logging, scheduling and monitoring. Red Hat's OpenShift is one of them, as well Weaveworks. Maybe try having a look at one such solution?
- psynapse 8y ago
  There is also the Canonical Distribution of Kubernetes (CDK). If you want to try a stripped down version on your local machine, kubernetes-core can be installed into LXD containers via conjure-up with a click of the fingers - nice for playing around.
wikibob 8y ago
It requires considerable operational experience and effort to run well on bare metal. Have you considered experimenting with a managed Kubernetes offering? Out of EKS, AKS, and GKE, I cannot recommend GKE highly enough. Even if you can’t use it for production, it’s highly worth setting up a prototype environment on GKE to see what it gives you. I believe they now have GPU support, including support for preemptible GPUs (much cheaper). Also GKE does a good job of staying up to date with releases. They are on 1.10, with support expected soon for 1.11. They _fully_ manage the upgrade of both the Kubernetes and etcd masters, and the worker nodes. As mentioned in the linked blog post, Kubernetes is a fast moving project, and to use it you should plan and allocate significant resources in your team to keeping up with the new releases. There are a large number of fixes and improvements since 1.8 and I would look very seriously at both upgrading, and changing your processes to allow you to stay closer to the current release version. The Kubernetes project does not, and has no current plans to, have a long term support release.

brianmcc 8y ago

Been looking for a decent intro write up for some time - this could be it!

auslander 8y ago

If you are on AWS, try ECS first, it is much simpler and has main features you need: HA, autoscaling, version control. When you'll mature from using Docker to high volume production, you should ditch containers at all, they are good for prototyping and testing, but not for production loads and production security.

wikibob 8y ago
ECS is simple to get started, but does not have a vast array of features that are necessary to run a service reliably at scale. It is possible to add many of these to ECS, but it’s left as an exercise to the reader to write up various API’s, cloudwatch, Lambda, etc. I can provide more specific details if there’s questions.
- auslander 8y ago
  Thank you. Yes, could you name the features ECS is missing for running at scale reliably, please?
  - shawabawa3 8y ago
    I've recently moved from ECS to Kubernetes, and much prefer kubernetes In general ECS is fine when you have 1 or 2 services, but once you have 10s or 100s it gets pretty unusable. Logging: While ECS ships everything to cloudwatch, actually accessing them is a nightmare and usually more effort than it's worth. It's very easy on k8s to get logs shipped from every container to elasticsearch and browse with kibana. Autoscaling: Possibly less of an issue now that fargate's a thing - but fargate's expensive. Kubernetes makes it easy to set up both node level autoscaling, and deployment level autoscaling Ingress: With ECS you have to set up loadbalancers for every service individually - in k8s once the controller's properly set up every service defines its own ingress, and the only manual change you have to do is create a DNS entry (only if you can't use a wildcard) Metrics: Prometheus is very ingrained in kubernetes, and you get a wealth of information from every service almost "for free" once you've installed it. And more: service meshes, secret control (through e.g. vault), declarative definitions, provider agnostic Kubernetes has a huge startup cost and learning curve, but is incredibly powerful once you're up and running

colonelpopcorn 8y ago

I really want to like Kubernetes, but going beyond the basics seems to require a way higher understanding of systems engineering than I currently have. Yes I know you can create container networks and stateful pods with attached storage, but how is always seemingly beyond me. Network and storage in distributed computing is hard and Kubernetes seems to be a slightly more magical bullet than Docker Swarm alone.

kgoutham93 8y ago
Completely agree... Going beyond the basics is really hard. But as I understand it's because of inadequate knowledge of advanced network/storage virtualization concepts. Any help on how to get started in those? As a side note I have decent knowledge of basic networking/storage.
outworlder 8y ago
Look up persistent volume claims and dynamic provisioning. If you cluster is properly configured, then you are one yaml file away from having that. By "configured properly" this means having the correct cloud provider set, otherwise it cannot talk to the APIs. To be fair, this stateful containers in general are a relatively new thing in K8s and support has been improving. Also, K8s is trying to do and abstract away a lot, it is more like a distributed operating system by itself. So it is more complicated than swarm.

mosselman 8y ago

What I find the hardest to figure out is how to properly deploy databases in kubernetes. What kind of volumes should I use and how do I configure them for production instead of some hello world situation?

djsumdog 8y ago
I think in ever docker ecosystem shop I've worked at, Databases are always on RDS. For large storage, we bought NAS solutions in the local data center. Our DC/OS cluster was primarily for things that didn't need to keep any sort of state. We had a few containers that needed NFS access and that was a lot of special work with the platform team, using labels to pin containers to certain nodes.
afroisalreadyin 8y ago
A database would normally be deployed as a stateful set, with the data stored in a volume, created and maintained by a PVC (persistent volume claim). This mechanism ensures that in the case of your pod dying and getting recreated on a different volume, it would see the same data as before, with the PVC getting mounted on the same pod. This mechanism is still one of the weaker links in the Kubernetes chain, in my experience, as the PVC feature is implemented completely differently on each platform, and is prone to bugs (see https://github.com/kubernetes/kubernetes/issues/60101 https://github.com/kubernetes/kubernetes/issues/60101, from Kubernetes version 1.7 on Azure, for example). Kelsey Hightower himself mentioned in a tweet that he prefers hosted solutions for persistency, and Kubernetes for stateless applications. Also, you should have a look at operators for managing applications on Kubernetes, such as this one for PostgreSQL: https://github.com/zalando-incubator/postgres-operator https://github.com/zalando-incubator/postgres-operator.
threeseed 8y ago
Depends on your database. Some like Cassandra are easier since it is multi-master and so you could just use local storage and if the node goes down then it's not the end of the world. There is an interesting solution for MySQL: https://vitess.io https://vitess.io which apparently is used by Youtube and Slack.

djsumdog 8y ago

I gave up on my own Kubernetes writeup a while back. I just had a lot of trouble with basic networking configuration, logging, etc. I've been at one shop with a large scale DC/OS installation. You can run a k8s scheduler on DC/OS, but by default it uses Marathon. DC/OS has it's own problems for sure, and both tools require a full time team of at least 3 people (we had 8~10) and there are a lot of things that will probably need to be customized for your shop (which labels to use, scripts to setup your ingress/egress points in AWS, HAproxy configuration or marathon-lb configuration .. which is just a haproxy container/wrapper), but I think I still prefer marathon. I briefly played with nomad and which I had spent more time with it. I know people from at least one startup around where I live using it in production. It seems to be a bit more minimal and potentially more sane. The thing I hate about all of these is there is no 1 to n scaling. For a simple project, I can't just setup one node with a minimal scheduler. DC/OS is going to cost you ~$120 a month for one non-redundant node: https://penguindreams.org/blog/installing-mesosphere-dcos-on-small-digital-ocean-droplets/ https://penguindreams.org/blog/installing-mesosphere-dcos-on... I hear people talk about minicube, but that's not something you can expand from one node to 100 right? You still have to build out a real k8s cluster at some point. All of these tools are just frontends around a scheduling and container engine (typically Docker and VMs) that track which containers are running where and track networking between nodes (and you often still have to chose and configure that networking layer .. weavenet, flannel, etc). I know someone will probably mention Rancer, and I should probably look at it again, but last time I looked I felt it was all point-n-click GUI and not enough command line flags (or at least not enough documented CLI) to really be used in an infrastructure as code fashion. I feel like there's still a big missing piece of the docker ecosystem, a really simple scheduler that can easily be stood up on new nodes and attach them to an existing cluster, and has a simply way of handling public IPs for web apps/haproxy containers. I know you can do this with K8s, DC/OS, etc. But there is a lot of prep work that has to be done first.

colonelpopcorn 8y ago
I have yet to see a good tutorial that shows an automated build of a kubernetes cluster. Yeah, you can use GKE, but that gets to be prohibitively expensive.

tobbyb 8y ago

We are working on a project with standard LXC containers [1] which tries to make orchestration and some of this stuff especially networking simpler. We support provisioning servers, building overlay networks with Vxlan, BGP & Wireguard, distributed storage and rolling out things like service discovery, load balancers and HA. It may be worth exploring for those struggling with some of the complexity around container deployments. At the minimum it will help you understand more about containers, networking and orchestration. [1] https://www.flockport.com https://www.flockport.com

rcconf 8y ago

I've been using Kubernetes on Azure and GCE recently and it's absolutely wonderful. I was able to setup an entire ecosystem from scratch in a week that scales well and can be managed in one location. When I first looked at Kubernetes, the complicated part was setting it up on a cluster. If you use Kubernetes on GCE, or Azure, you don't have to do that step, everything else is ready to go for you! - Automatic scaling of your application - Service discovery - Secrets and config management - Logging in one central dashboard - Able to deploy various complicated, distributed, pieces of software very easily using Helm (Jenkins, Kafka, Grafana + Prometheus) - Able to add new nodes to the cluster easily - Health checks and automatic restarts - Able to deploy any container to your cluster in a really simple way (if you look at Deployments, it's really simple.) - Switch between cloud providers and still maintain the same workflow. I won't ever touch Ansible again, I really prefer the Kubernetes way of handling operations (it's like a live organism instead of something you apply changes to.) Also, the entire argument that you probably don't need Kubernetes because your organization doesn't have 10s, or 100s of nodes just doesn't make sense after using it. Having a Kubernetes cluster with 3 nodes is 100% worth it, even for rather simple applications in my opinion. The benefits are just way too good.

ramoz 8y ago
How was your networking experience? Any exp running within Enterprise VPN? Private image repos in the network? And if you needed to leave GKE to another provider or packaged , how would that look? Thanks for any answers!
TuringNYC 8y ago
I'm glad you mentioned Azure and GCE -- I started off with minikube locally and things didnt really become magical until i started using a kops-deployed AWS-EC2 cluster and saw the magic of elastic scaling, ease of deployment, and migrate-ability. PS - Could you share which books/websites/resources you used to get up to speed to the point where you're at now?
- chadmeister 8y ago
  Would you mind sharing links to any of the resources you used to get yourself going with kops on AWS? I'm in the midst of setting up my first cluster as we speak.

merinowool 8y ago

I wish I could find a tutorial for bare metal: - how to setup cert creation for fqdn with Cloudflare - storage - ingress to support multiple ips glued to different nodes (so if service gets IP x it gets routed through node z that has this external IP) I spent 6 months trying to do that and no luck.

joshschreuder 8y ago

Scott McCloud, author of the great book Understanding Comics, also did a comic book introduction to Kubernetes: https://cloud.google.com/kubernetes-engine/kubernetes-comic/ https://cloud.google.com/kubernetes-engine/kubernetes-comic/

deleted 8y ago
[deleted]
emmelaich 8y ago
..including interactive demo start! Nice. PS. McCloud? I sense nominative determinism at play.

ohiovr 8y ago

What is the typical use case for clustered applications? What size organization really needs it? I understand that a simple nginx static site can accept thousands of simultaneous connections per host. That sounds pretty huge to me. If you were to sell kubernetes based solutions who would you consider selling to? What makes kubernetes fundamentally superior to docker swarm?