14 ms·

As Kubernetes grows, a startup ecosystem develops in its wake

184 points by banderon 8y ago

epberry 8y ago
I didn't realize all the pieces of the cloud stack that CNCF was working on and how each piece gives you more flexibility to move between cloud providers. Kubernetes and rkt replace servers and orchestration of course but you also have Open Policy Agent and Spire to replace IAM, Envoy and Jaeger replace X-ray, VPC, Security Groups, Rook replaces S3, etc. It's kind of interesting because Google and Microsoft have opened a new front against Amazon to counter their strategy of locking you in with services. If software from CNCF is good enough to be used in place of these services on each cloud provider then you can nullify AWS's advantage there and Google can lure you in with ML and Microsoft with their enterprise experience. But I'm not sure these projects can make progress faster than AWS can release and update their fantastic array of services. Kubernetes being the obvious exception (when is EKS GA?!?!)
atombender 8y ago
Kubernetes is one of those rare cases where the industry is rallying around a technologically sound platform that emerged from the trenches, and not something that was designed to create jobs for consultants. It's great. Right now, there's a bunch of fairly traditional players such as Red Hat, but none of them are in a position to mess it up. What I'm waiting for, though, is for a big player to do a modern, clean "Kubernetes first" cloud offering. We're currently on Google Kubernetes Engine, and I'm disappointed in the lack of integration throughout Cloud Platform. GCP is great in many areas, but the Kubernetes offering is clearly an afterthought. As an example, if you create an ingress, this creates a bunch of load balancer objects (forwarding rules, URL maps, backends). But these are all given obscure names (k8s-um-production-zippediboop-f8da5e6c92f38300), and none of the Kubernetes labels transfer over. Same with disks; GCE disks don't even have labels, so they put a bunch of JSON in the "description", which of course cannot be filtered or queried. Similar things happen with VMs and other things; the "mapping" of Kubernetes to GCP concepts is basically either lossy or non-existent. Many other similar weaknesses exist: Cloud SQL (if you want to use Cloud SQL, you have to manually run the Cloud SQL Proxy as a sidecar), GCP service accounts (different from Kubernetes service accounts!), etc. GKE is solid, but everything seems like it's been put together without any over-arching design; GKE is just another thing buried in a somewhat ramshackle jungle of different services. There's an opportunity for a vendor to come in and offer a platform where you start with Kubernetes. In particular, this means that the platform should be manageable from Kubernetes, through the use of CRDs and pluggable controllers. For example, I should be able to define a persistent disk using a Kubernetes manifest, instead of having to go through some web UI or CLI ("gcloud compute disk create" or whatever). That said, it's hard to compete with GCP at this point. Whoever wants to compete in this space have to compete not just with GKE, but with the technical awesomeness of GCP.
- navaati 8y ago
  Well RedHat's OpenShift.com is kinda that, have a look at it !
  - lobster_johnson 8y ago
    Maybe. I'm skeptical of Red Hat. The openshift.com web site, which is definitely not geared toward engineers, doesn't give me any confidence that this is what I'm after. Not that Google is better here, but I already know what GCP provides; something that wants to compete with GCP/GKE really needs to explain why they're a viable competitor. (And to people who design these things: If you need to have a "Products" dropdown filled with unexplained product names, you're doing something wrong.)
    - ghaff 8y ago
      A lot of people come to websites looking for specific products. When you don’t offer an obvious path to what they’re looking for, they get frustrated. I know I do.
    - organsnyder 8y ago
      Red Hat's marketing is definitely a mess (I say this as an enterprise Red Hat customer in the midst of an OpenShift roll-out), but their engineering is top-notch. Many pieces of k8s originated in OpenShift, and Red Hat is heavily involved in k8s development.
    - jacques_chester 8y ago
      I work for Pivotal, we directly compete with Red Hat. Look: they're Red Hat, I mean c'mon now. If you don't think they have great engineers then I'd hate to hear what you think of the rest of us. These websites are not usually intended for you or I. They're intended to get the interest of the cheque-signers. Enterprise software is a very particular world and it has its particular cadences and norms and approaches. Red Hat understands it well. So does Pivotal. That's why we see so much of each other at a number of places where folks are looking to adopt modern platforms.
ksec 8y ago
Does anyone know if Apple uses Kubernetes in their cloud?
- puzzle 8y ago
  I don't know that for a fact, but I have seen Apple people at KubeCon, for what it's worth.
- theptip 8y ago
  Siri, at least, operated on Mesos last I heard: https://www.infoq.com/news/2015/05/mesos-powers-apple-siri https://www.infoq.com/news/2015/05/mesos-powers-apple-siri I would be surprised if they haven't done a k8s PoC at the very least though.
- josegonzalez 8y ago
  They use Nomad from Hashicorp in certain parts of their cloud.
  - iampims 8y ago
    Do you have more info on this?
- nameless912 8y ago
  From snooping around at KubeCon NA last December, my impression is that Apple is doing something involving GPU computing on top of Kubernetes. My guess would be machine learning, but they were characteristically hush-hush about it.
- dewey 8y ago
  https://jobs.apple.com/us/search?job=113384172&openJobId=113384172#&openJobId=113384172 https://jobs.apple.com/us/search?job=113384172&openJobId=113... >Big extra points if you have experience with orchestration technologies like: - Mesos/Mesosphere - Kubernetes - Docker [Swarm]"
throwaway5752 8y ago
https://techcrunch.com/2018/05/07/mesosphere-hauls-in-125-m-series-d-investment/ https://techcrunch.com/2018/05/07/mesosphere-hauls-in-125-m-... seems apt and got very little attention in https://news.ycombinator.com/item?id=17012856 https://news.ycombinator.com/item?id=17012856, given the size of the round.
mosselman 8y ago
Recently I have been playing with kubernetes through Rancher. I like how easy it is to install kubernetes through rancher, essentially I ran 2 commands: 1. on the machine that runs rancher and 2. on the first node in my cluster. After a few days of playing I set up lets encrypt with load balancing, a running app (Rails). A remaining issue are persistent volumes and how truly persistent these are. I haven't found out yet which solution I should pick for this. Longhorn is a Rancher product, which is probably what I will read more about now, but I cannot be sure. There are so many concepts and terminology that you need to figure out. Having Rancher in between is not helping me get a hang of kubernetes itself more purely of course. A while back I was playing with Docker Swarm and I must say that I like Docker Swarm better in the sense that it feels closer to the source and because it is built into Docker. I get a feeling however that Kubernetes is where the future is so learning more about Docker Swarm is probably less worthwhile.
anoncoward1234 8y ago
Kubernetes is cool and all, but there needs to be a lot of simplification for it to be "nice to use". Essentially the problem is that it's the "opposite" of Golang - the number of ways to do the same thing is massive leading to huge numbers of headaches in trying to get things done. Recently, I spun up a simple pod-to-pod communication example but I found it pretty difficult. If you look up cluster networking in Kubernetes (https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-achieve-this https://kubernetes.io/docs/concepts/cluster-administration/n...) you'll find a whole fire hose of different options from ingress to calico to fabric and on and on. This was what it took for me to try and rubber ducky my way to getting networking to work on Kubernetes, and in the end I had to get help from a friend at work (https://stackoverflow.com/questions/50195896/how-do-i-get-one-pod-to-network-to-another-pod-in-kubernetes-simple https://stackoverflow.com/questions/50195896/how-do-i-get-on...). It may be better than what came before, but it's not great.
- outworlder 8y ago
  Why don't you use something like Kops? It will create (and manage) the cluster for you.
- shaklee3 8y ago
  Kops, kubeadm, rancher, kubespray all do what you want, but differently depending on your needs. What were you looking for?
  - anoncoward1234 8y ago
    EXACTLY
    - shaklee3 8y ago
      I don't see your point, but I should have clarified more: Kops == AWS install Rancher == Easy small cluster on-prem or private cloud. GUI push-button setup kubeadm == CLI tool to set up a cluster manually with basic settings. Not very flexible, but can get a cluster working in minutes. kubespray == Ansible playbooks for setting up k8s cluster. The most powerful of the bunch, especially if you're familiar with ansible. This is the preferred way to run it in production I believe, along with customizations on top.
akhilcacharya 8y ago
Honestly, the K8S ecosystem is so wide now I'd love to see an up-to-date explainer piecing it all together. What folks need, what they don't, etc.
- dankohn1 8y ago
  Try the cloud native trail map linked at the top of https://l.cncf.io https://l.cncf.io
gerdesj 8y ago
How do you pronounce Kubernetes?
- berg01 8y ago
  https://github.com/kubernetes/kubernetes/issues/44308 https://github.com/kubernetes/kubernetes/issues/44308
- shaklee3 8y ago
  Koo-ber-net-tees
- emmelaich 8y ago
  κυβερνητική (almost) It's actually the same root word as cybernetics. Meaning governance. Hence tools named like Helm etc. I believe it's the root of words like 'captain' too. Maybe even related to Caesar / Kaiser via Latin. https://en.wikipedia.org/wiki/Cybernetics#Etymology https://en.wikipedia.org/wiki/Cybernetics#Etymology
  - shaklee3 8y ago
    It means navigator in Greek.
ex3ndr 8y ago
Does anyone knows a very good CD for k8s? I found that only bitbucket deployments is a good one in terms of simplicity of managing deployments (just like Heroku). Most vendors forces me to use their crappy CI solution for doing CD. Why they want me to migrate to your (very limited) build system? I can pay you for a fancy dashboard, but please, allow me to use whatever i need to build my software. There are many options on the market for CI and you just can't build them for everybody. But good CD is a way to go and easier to manage. Just give me a hook for registering for a new build (say docker image versions) and help me manage this stuff. I have tried Spinnakle, but it is too fragile for me - there are no simple way to install it (k8s even easier!) and UI is too bad for a small project. Good CD is still missing in k8s ecosystem.
- outworlder 8y ago
  Closest I've got was Spinnaker. Support for K8s was very hit and miss (some concepts did not translate properly), but it is getting much better now.
  - techcofounder 8y ago
    I assume you mean the Spinnaker/K8s v1 provider abstracted away the K8s primatives. The new v2 provider allows you to deploy manifests and supports all K8s objects.
- shaabanban 8y ago
  Gitlab CI has excellent support for k8s but it requires buy in to the whole ecosystem to be truly useful...
- 0x6c6f6c 8y ago
  I'm awaiting jenkins-x's maturation. I'm looking at some issues for fixing myself but there's still plenty to do. Support is there for GCS and AWS iirc, but it's still very much beta.
- lobster_johnson 8y ago
  We used Drone [1] from a while. It's almost exactly what I want in a CI/CD system, and it runs pretty well on Kubernetes. On the minus side, the development has been slow, with major breaking changes happening along the way. And in the latest version, the open-source version has been a bit hobbled. In particular, it doesn't support global secrets, so every project to be built has to be created and managed separately. This is not just if it needs build secrets (such as to access private Go packages, NPM modules, Ruby gems, etc.), but it's obviously also required to authenticate with a container registry. We decided to skip that since it means duplicating the same secrets for every single application. We actually reached out to the Drone guys to ask about enterprise pricing, but they didn't respond (!). (At the moment we're back to building with Semaphore, which is a hosted solution similar to Travis. It's an old-hat CI system that spins up VMs, so it's slow and awkward to work with when it comes to Docker. But it's working okay at the moment.) I've heard good things about GoCD [2], though. It's next on my list to investigate. [1] https://drone.io https://drone.io [2] https://www.gocd.org https://www.gocd.org
ryanSrich 8y ago
This is true for startups new and old. At Datica (where I work), we started in 2014 with a bespoke container orchestration layer. This powered our HITRUST certified Platform. Think Heroku for regulated industries (like Healthcare). After years of hardship trying to keep up with the market we finally decided to go all in on K8s. Kubernetes gives us the flexibility and community to focus on the compliance and security layer, while not having to build a Platform in house. Until now, no other open source solution had given us this type of flexibility. We're still working toward a GA release, but the speed at which we've been able to move has been incredible.
drej 8y ago
It’s always nice when there is some sort of consensus in the industry, however, there is one thing I worry about. Now that there is a usable solution for complex orchestration, many newcomers will consider it the goto solution, regardless of scale. You need nginx and some python scripts? Kubernetes! You have 100 daily visitors? Kubernetes! It’s not really their fault, it’s just a sad consequence of a convenient cloud solution.
- jrs95 8y ago
  Part of this is also people jumping to microservices when they really have no need for them. When you have less than 5 developers and very little traffic...it's just absurd. But people are doing it anyways. Because "monoliths" are "bad". And honestly, I wouldn't be surprised if a lot of these microservice architectures end up totally botched and creating a ton of headaches without delivering much benefit.
  - netheril96 8y ago
    Another part is claiming Kubernetes experiences on your resume.
- aphextron 8y ago
  Is that really a problem though? The entire history of software development is just building on one layer of abstraction over another. Is it sad that people these days don't write their own web server for new projects?
  - kerneltime 8y ago
    The point is not to rewrite from scratch.. the point is that your judgement to use a piece of technology should come from your understanding of the problem and direction and not based on what is a legitimately a hot topic in a context that does not apply to you.. in other words do not let google search results be the bar you meet in solving your problems..
  - jrs95 8y ago
    A lot of people are only adopting Kubernetes because they think they will benefit from Dockerized microservices when they don't even have most of the problems that solves. There's a lot of extra development and operations effort going into things for no real reason other than mistaking this technology for a silver bullet.
daxfohl 8y ago
There needs to be a digitalocean or heroku for containerization. Currently to start a k8s whatever you need to go through various manual rigamaroles and spend a lot of money, or go through the big lockin providers and spend a lot of money. For those of us not backed by venture capital and not charging SV ex-googler rates to our clients, we need something to say "we'll host you on this git push containerization thing, it'll be cheap and easy, and we'll expand when the time comes, and that'll be cheap and easy too".
- praveenweb 8y ago
  Check out https://hasura.io https://hasura.io - you get git push based deployment on a kubernetes cluster.
- iampims 8y ago
  https://hyper.sh https://hyper.sh aimed to be a CaaS (Container as a service) and Joyent’s Triton as well. It’s an interesting premise, but from the outside both don't have much traction.
- pi-squared 8y ago
  Actually DO just announced k8s for later this year: https://blog.digitalocean.com/introducing-digitalocean-kubernetes/ https://blog.digitalocean.com/introducing-digitalocean-kuber...
- bonesss 8y ago
  The Azure Container Instances seems like it's aiming to be a "heroku for containers" -- https://azure.microsoft.com/en-us/services/container-instances/ https://azure.microsoft.com/en-us/services/container-instanc...
- geku 8y ago
  We are working on https://www.KubeBox.com https://www.KubeBox.com to provide simple Kubernetes clusters for an affordable price. For $36/month (8GB Ram, 2vCPU) you get a single node if you just want to have the Kubernetes APIs. Maybe introducing smaller node plans later in the year. Unfortunately, not much info on our website but please sign up if you are interested.
indescions_2018 8y ago
Really glad to see K8 getting this level of investment. Even if it will take a month to explore all the links in the CNCF landscape ;) Just want to remind anyone who hasn't tried K8. You can run MiniKube locally right on your laptop to get a taste of its power (and complexity): https://kubernetes.io/docs/getting-started-guides/minikube/ https://kubernetes.io/docs/getting-started-guides/minikube/ For example, a 3-node Redis Enterprise cluster, all run locally: https://redislabs.com/blog/local-kubernetes-development-using-minikube-redis-enterprise/ https://redislabs.com/blog/local-kubernetes-development-usin... Currently using GCloud and Stackdriver monitoring, but a few of the tools I am excited about include: Prometheus / Granafa / KSonnet KubeFlow ML https://www.youtube.com/watch?v=I6iMznIYwM8 https://www.youtube.com/watch?v=I6iMznIYwM8 Istio, for programmatic routing https://istio.io/ https://istio.io/ And, Agones, for game hosting https://agones.dev/ https://agones.dev/
hermitdev 8y ago
I'm curious: how does containerization work for systems that utilize kernel-bypass? Is it even possible? I did a (admittedly short) search for kubernetes and kernel bypass, and the only thing that seem remotely relevant was [0], however it didn't indicate whether they work together. For background, I work for a Dark Pool Alternative Trading System, and we currently utilize kernel bypass for all of our networking using Solarflare NICs & openonload [1]. In the same vein, curious how containers work with CPU shielding and pinning threads to specific cores. Is it possible, and how do multiple containers on the same box interact in that regard. Do they need to be quasi-aware of each other so as to not pin a thread to the same core? I'd greatly appreciate if anyone with experience with containers can answer these questions. I'm genuinely curious, but it's not worth researching further if there's no solution that can handle these strict requirements (e.g. it's a non-starter if containers increase latency). [0] https://thenewstack.io/life-post-container-world/ https://thenewstack.io/life-post-container-world/ [1] http://openonload.org/ http://openonload.org/
- hunter_n 8y ago
  I've not had direct experience with it but do recall Intel covering their work with DPDK and Core pinning. I think they got something like 96% line rate on 25GbE. - https://networkbuilders.intel.com/network-technologies/container-experience-kits https://networkbuilders.intel.com/network-technologies/conta...
alexnewman 8y ago
I think the real advantage of k8s is the ease to scale bare metal. This just isn't that important for the next 5 years or so. On the other hand, azure and gke make k8s easy enough if you are already using docker. Containers on the hand, the developer experience is still very mixed. OSX still doesn't have great container support and I honestly can only guess how well windows can handle my docker-compose workflow. One great k8s tool I like is kompose. It allows our devs a very similar interface around secrets, networks, volumes, etc.

184 points by banderon 8y ago

I didn't realize all the pieces of the cloud stack that CNCF was working on and how each piece gives you more flexibility to move between cloud providers. Kubernetes and rkt replace servers and orchestration of course but you also have Open Policy Agent and Spire to replace IAM, Envoy and Jaeger replace X-ray, VPC, Security Groups, Rook replaces S3, etc. It's kind of interesting because Google and Microsoft have opened a new front against Amazon to counter their strategy of locking you in with services. If software from CNCF is good enough to be used in place of these services on each cloud provider then you can nullify AWS's advantage there and Google can lure you in with ML and Microsoft with their enterprise experience. But I'm not sure these projects can make progress faster than AWS can release and update their fantastic array of services. Kubernetes being the obvious exception (when is EKS GA?!?!)

atombender 8y ago

Kubernetes is one of those rare cases where the industry is rallying around a technologically sound platform that emerged from the trenches, and not something that was designed to create jobs for consultants. It's great. Right now, there's a bunch of fairly traditional players such as Red Hat, but none of them are in a position to mess it up. What I'm waiting for, though, is for a big player to do a modern, clean "Kubernetes first" cloud offering. We're currently on Google Kubernetes Engine, and I'm disappointed in the lack of integration throughout Cloud Platform. GCP is great in many areas, but the Kubernetes offering is clearly an afterthought. As an example, if you create an ingress, this creates a bunch of load balancer objects (forwarding rules, URL maps, backends). But these are all given obscure names (k8s-um-production-zippediboop-f8da5e6c92f38300), and none of the Kubernetes labels transfer over. Same with disks; GCE disks don't even have labels, so they put a bunch of JSON in the "description", which of course cannot be filtered or queried. Similar things happen with VMs and other things; the "mapping" of Kubernetes to GCP concepts is basically either lossy or non-existent. Many other similar weaknesses exist: Cloud SQL (if you want to use Cloud SQL, you have to manually run the Cloud SQL Proxy as a sidecar), GCP service accounts (different from Kubernetes service accounts!), etc. GKE is solid, but everything seems like it's been put together without any over-arching design; GKE is just another thing buried in a somewhat ramshackle jungle of different services. There's an opportunity for a vendor to come in and offer a platform where you start with Kubernetes. In particular, this means that the platform should be manageable from Kubernetes, through the use of CRDs and pluggable controllers. For example, I should be able to define a persistent disk using a Kubernetes manifest, instead of having to go through some web UI or CLI ("gcloud compute disk create" or whatever). That said, it's hard to compete with GCP at this point. Whoever wants to compete in this space have to compete not just with GKE, but with the technical awesomeness of GCP.

navaati 8y ago
Well RedHat's OpenShift.com is kinda that, have a look at it !
- lobster_johnson 8y ago
  Maybe. I'm skeptical of Red Hat. The openshift.com web site, which is definitely not geared toward engineers, doesn't give me any confidence that this is what I'm after. Not that Google is better here, but I already know what GCP provides; something that wants to compete with GCP/GKE really needs to explain why they're a viable competitor. (And to people who design these things: If you need to have a "Products" dropdown filled with unexplained product names, you're doing something wrong.)
  - ghaff 8y ago
    A lot of people come to websites looking for specific products. When you don’t offer an obvious path to what they’re looking for, they get frustrated. I know I do.
  - organsnyder 8y ago
    Red Hat's marketing is definitely a mess (I say this as an enterprise Red Hat customer in the midst of an OpenShift roll-out), but their engineering is top-notch. Many pieces of k8s originated in OpenShift, and Red Hat is heavily involved in k8s development.
  - jacques_chester 8y ago
    I work for Pivotal, we directly compete with Red Hat. Look: they're Red Hat, I mean c'mon now. If you don't think they have great engineers then I'd hate to hear what you think of the rest of us. These websites are not usually intended for you or I. They're intended to get the interest of the cheque-signers. Enterprise software is a very particular world and it has its particular cadences and norms and approaches. Red Hat understands it well. So does Pivotal. That's why we see so much of each other at a number of places where folks are looking to adopt modern platforms.

ksec 8y ago

Does anyone know if Apple uses Kubernetes in their cloud?

puzzle 8y ago
I don't know that for a fact, but I have seen Apple people at KubeCon, for what it's worth.
theptip 8y ago
Siri, at least, operated on Mesos last I heard: https://www.infoq.com/news/2015/05/mesos-powers-apple-siri https://www.infoq.com/news/2015/05/mesos-powers-apple-siri I would be surprised if they haven't done a k8s PoC at the very least though.
josegonzalez 8y ago
They use Nomad from Hashicorp in certain parts of their cloud.
- iampims 8y ago
  Do you have more info on this?
nameless912 8y ago
From snooping around at KubeCon NA last December, my impression is that Apple is doing something involving GPU computing on top of Kubernetes. My guess would be machine learning, but they were characteristically hush-hush about it.
dewey 8y ago
https://jobs.apple.com/us/search?job=113384172&openJobId=113384172#&openJobId=113384172 https://jobs.apple.com/us/search?job=113384172&openJobId=113... >Big extra points if you have experience with orchestration technologies like: - Mesos/Mesosphere - Kubernetes - Docker [Swarm]"

throwaway5752 8y ago

https://techcrunch.com/2018/05/07/mesosphere-hauls-in-125-m-series-d-investment/ https://techcrunch.com/2018/05/07/mesosphere-hauls-in-125-m-... seems apt and got very little attention in https://news.ycombinator.com/item?id=17012856 https://news.ycombinator.com/item?id=17012856, given the size of the round.

mosselman 8y ago

Recently I have been playing with kubernetes through Rancher. I like how easy it is to install kubernetes through rancher, essentially I ran 2 commands: 1. on the machine that runs rancher and 2. on the first node in my cluster. After a few days of playing I set up lets encrypt with load balancing, a running app (Rails). A remaining issue are persistent volumes and how truly persistent these are. I haven't found out yet which solution I should pick for this. Longhorn is a Rancher product, which is probably what I will read more about now, but I cannot be sure. There are so many concepts and terminology that you need to figure out. Having Rancher in between is not helping me get a hang of kubernetes itself more purely of course. A while back I was playing with Docker Swarm and I must say that I like Docker Swarm better in the sense that it feels closer to the source and because it is built into Docker. I get a feeling however that Kubernetes is where the future is so learning more about Docker Swarm is probably less worthwhile.

anoncoward1234 8y ago

Kubernetes is cool and all, but there needs to be a lot of simplification for it to be "nice to use". Essentially the problem is that it's the "opposite" of Golang - the number of ways to do the same thing is massive leading to huge numbers of headaches in trying to get things done. Recently, I spun up a simple pod-to-pod communication example but I found it pretty difficult. If you look up cluster networking in Kubernetes (https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-achieve-this https://kubernetes.io/docs/concepts/cluster-administration/n...) you'll find a whole fire hose of different options from ingress to calico to fabric and on and on. This was what it took for me to try and rubber ducky my way to getting networking to work on Kubernetes, and in the end I had to get help from a friend at work (https://stackoverflow.com/questions/50195896/how-do-i-get-one-pod-to-network-to-another-pod-in-kubernetes-simple https://stackoverflow.com/questions/50195896/how-do-i-get-on...). It may be better than what came before, but it's not great.

outworlder 8y ago
Why don't you use something like Kops? It will create (and manage) the cluster for you.
shaklee3 8y ago
Kops, kubeadm, rancher, kubespray all do what you want, but differently depending on your needs. What were you looking for?
- anoncoward1234 8y ago
  EXACTLY
  - shaklee3 8y ago
    I don't see your point, but I should have clarified more: Kops == AWS install Rancher == Easy small cluster on-prem or private cloud. GUI push-button setup kubeadm == CLI tool to set up a cluster manually with basic settings. Not very flexible, but can get a cluster working in minutes. kubespray == Ansible playbooks for setting up k8s cluster. The most powerful of the bunch, especially if you're familiar with ansible. This is the preferred way to run it in production I believe, along with customizations on top.

akhilcacharya 8y ago

Honestly, the K8S ecosystem is so wide now I'd love to see an up-to-date explainer piecing it all together. What folks need, what they don't, etc.

dankohn1 8y ago
Try the cloud native trail map linked at the top of https://l.cncf.io https://l.cncf.io

gerdesj 8y ago

How do you pronounce Kubernetes?

berg01 8y ago
https://github.com/kubernetes/kubernetes/issues/44308 https://github.com/kubernetes/kubernetes/issues/44308
shaklee3 8y ago
Koo-ber-net-tees
emmelaich 8y ago
κυβερνητική (almost) It's actually the same root word as cybernetics. Meaning governance. Hence tools named like Helm etc. I believe it's the root of words like 'captain' too. Maybe even related to Caesar / Kaiser via Latin. https://en.wikipedia.org/wiki/Cybernetics#Etymology https://en.wikipedia.org/wiki/Cybernetics#Etymology
- shaklee3 8y ago
  It means navigator in Greek.

ex3ndr 8y ago

Does anyone knows a very good CD for k8s? I found that only bitbucket deployments is a good one in terms of simplicity of managing deployments (just like Heroku). Most vendors forces me to use their crappy CI solution for doing CD. Why they want me to migrate to your (very limited) build system? I can pay you for a fancy dashboard, but please, allow me to use whatever i need to build my software. There are many options on the market for CI and you just can't build them for everybody. But good CD is a way to go and easier to manage. Just give me a hook for registering for a new build (say docker image versions) and help me manage this stuff. I have tried Spinnakle, but it is too fragile for me - there are no simple way to install it (k8s even easier!) and UI is too bad for a small project. Good CD is still missing in k8s ecosystem.

outworlder 8y ago
Closest I've got was Spinnaker. Support for K8s was very hit and miss (some concepts did not translate properly), but it is getting much better now.
- techcofounder 8y ago
  I assume you mean the Spinnaker/K8s v1 provider abstracted away the K8s primatives. The new v2 provider allows you to deploy manifests and supports all K8s objects.
shaabanban 8y ago
Gitlab CI has excellent support for k8s but it requires buy in to the whole ecosystem to be truly useful...
0x6c6f6c 8y ago
I'm awaiting jenkins-x's maturation. I'm looking at some issues for fixing myself but there's still plenty to do. Support is there for GCS and AWS iirc, but it's still very much beta.
lobster_johnson 8y ago
We used Drone [1] from a while. It's almost exactly what I want in a CI/CD system, and it runs pretty well on Kubernetes. On the minus side, the development has been slow, with major breaking changes happening along the way. And in the latest version, the open-source version has been a bit hobbled. In particular, it doesn't support global secrets, so every project to be built has to be created and managed separately. This is not just if it needs build secrets (such as to access private Go packages, NPM modules, Ruby gems, etc.), but it's obviously also required to authenticate with a container registry. We decided to skip that since it means duplicating the same secrets for every single application. We actually reached out to the Drone guys to ask about enterprise pricing, but they didn't respond (!). (At the moment we're back to building with Semaphore, which is a hosted solution similar to Travis. It's an old-hat CI system that spins up VMs, so it's slow and awkward to work with when it comes to Docker. But it's working okay at the moment.) I've heard good things about GoCD [2], though. It's next on my list to investigate. [1] https://drone.io https://drone.io [2] https://www.gocd.org https://www.gocd.org

ryanSrich 8y ago

This is true for startups new and old. At Datica (where I work), we started in 2014 with a bespoke container orchestration layer. This powered our HITRUST certified Platform. Think Heroku for regulated industries (like Healthcare). After years of hardship trying to keep up with the market we finally decided to go all in on K8s. Kubernetes gives us the flexibility and community to focus on the compliance and security layer, while not having to build a Platform in house. Until now, no other open source solution had given us this type of flexibility. We're still working toward a GA release, but the speed at which we've been able to move has been incredible.

drej 8y ago

It’s always nice when there is some sort of consensus in the industry, however, there is one thing I worry about. Now that there is a usable solution for complex orchestration, many newcomers will consider it the goto solution, regardless of scale. You need nginx and some python scripts? Kubernetes! You have 100 daily visitors? Kubernetes! It’s not really their fault, it’s just a sad consequence of a convenient cloud solution.

jrs95 8y ago
Part of this is also people jumping to microservices when they really have no need for them. When you have less than 5 developers and very little traffic...it's just absurd. But people are doing it anyways. Because "monoliths" are "bad". And honestly, I wouldn't be surprised if a lot of these microservice architectures end up totally botched and creating a ton of headaches without delivering much benefit.
- netheril96 8y ago
  Another part is claiming Kubernetes experiences on your resume.
aphextron 8y ago
Is that really a problem though? The entire history of software development is just building on one layer of abstraction over another. Is it sad that people these days don't write their own web server for new projects?
- kerneltime 8y ago
  The point is not to rewrite from scratch.. the point is that your judgement to use a piece of technology should come from your understanding of the problem and direction and not based on what is a legitimately a hot topic in a context that does not apply to you.. in other words do not let google search results be the bar you meet in solving your problems..
- jrs95 8y ago
  A lot of people are only adopting Kubernetes because they think they will benefit from Dockerized microservices when they don't even have most of the problems that solves. There's a lot of extra development and operations effort going into things for no real reason other than mistaking this technology for a silver bullet.

daxfohl 8y ago

There needs to be a digitalocean or heroku for containerization. Currently to start a k8s whatever you need to go through various manual rigamaroles and spend a lot of money, or go through the big lockin providers and spend a lot of money. For those of us not backed by venture capital and not charging SV ex-googler rates to our clients, we need something to say "we'll host you on this git push containerization thing, it'll be cheap and easy, and we'll expand when the time comes, and that'll be cheap and easy too".

praveenweb 8y ago
Check out https://hasura.io https://hasura.io - you get git push based deployment on a kubernetes cluster.
iampims 8y ago
https://hyper.sh https://hyper.sh aimed to be a CaaS (Container as a service) and Joyent’s Triton as well. It’s an interesting premise, but from the outside both don't have much traction.
pi-squared 8y ago
Actually DO just announced k8s for later this year: https://blog.digitalocean.com/introducing-digitalocean-kubernetes/ https://blog.digitalocean.com/introducing-digitalocean-kuber...
bonesss 8y ago
The Azure Container Instances seems like it's aiming to be a "heroku for containers" -- https://azure.microsoft.com/en-us/services/container-instances/ https://azure.microsoft.com/en-us/services/container-instanc...
geku 8y ago
We are working on https://www.KubeBox.com https://www.KubeBox.com to provide simple Kubernetes clusters for an affordable price. For $36/month (8GB Ram, 2vCPU) you get a single node if you just want to have the Kubernetes APIs. Maybe introducing smaller node plans later in the year. Unfortunately, not much info on our website but please sign up if you are interested.

indescions_2018 8y ago

Really glad to see K8 getting this level of investment. Even if it will take a month to explore all the links in the CNCF landscape ;) Just want to remind anyone who hasn't tried K8. You can run MiniKube locally right on your laptop to get a taste of its power (and complexity): https://kubernetes.io/docs/getting-started-guides/minikube/ https://kubernetes.io/docs/getting-started-guides/minikube/ For example, a 3-node Redis Enterprise cluster, all run locally: https://redislabs.com/blog/local-kubernetes-development-using-minikube-redis-enterprise/ https://redislabs.com/blog/local-kubernetes-development-usin... Currently using GCloud and Stackdriver monitoring, but a few of the tools I am excited about include: Prometheus / Granafa / KSonnet KubeFlow ML https://www.youtube.com/watch?v=I6iMznIYwM8 https://www.youtube.com/watch?v=I6iMznIYwM8 Istio, for programmatic routing https://istio.io/ https://istio.io/ And, Agones, for game hosting https://agones.dev/ https://agones.dev/

hermitdev 8y ago

I'm curious: how does containerization work for systems that utilize kernel-bypass? Is it even possible? I did a (admittedly short) search for kubernetes and kernel bypass, and the only thing that seem remotely relevant was [0], however it didn't indicate whether they work together. For background, I work for a Dark Pool Alternative Trading System, and we currently utilize kernel bypass for all of our networking using Solarflare NICs & openonload [1]. In the same vein, curious how containers work with CPU shielding and pinning threads to specific cores. Is it possible, and how do multiple containers on the same box interact in that regard. Do they need to be quasi-aware of each other so as to not pin a thread to the same core? I'd greatly appreciate if anyone with experience with containers can answer these questions. I'm genuinely curious, but it's not worth researching further if there's no solution that can handle these strict requirements (e.g. it's a non-starter if containers increase latency). [0] https://thenewstack.io/life-post-container-world/ https://thenewstack.io/life-post-container-world/ [1] http://openonload.org/ http://openonload.org/

hunter_n 8y ago
I've not had direct experience with it but do recall Intel covering their work with DPDK and Core pinning. I think they got something like 96% line rate on 25GbE. - https://networkbuilders.intel.com/network-technologies/container-experience-kits https://networkbuilders.intel.com/network-technologies/conta...

alexnewman 8y ago

I think the real advantage of k8s is the ease to scale bare metal. This just isn't that important for the next 5 years or so. On the other hand, azure and gke make k8s easy enough if you are already using docker. Containers on the hand, the developer experience is still very mixed. OSX still doesn't have great container support and I honestly can only guess how well windows can handle my docker-compose workflow. One great k8s tool I like is kompose. It allows our devs a very similar interface around secrets, networks, volumes, etc.