6 ms·

I completely agree. The current trend of orchestration all sells a story of hyperscale like GIFEE. That is not how most people do or should run there deploymen

I completely agree. The current trend of orchestration all sells a story of hyperscale like GIFEE. That is not how most people do or should run there deployments (I'm talking about the vast majority outside SV). The operational complexity today is too high. Kubernetes is a big business system, it caters to enterprise. If you aren't an enterprise I would caution you against k8s.

mwcampbell 8y ago
What do you think most people should use instead of k8s? Or is there not a good answer for that yet? Do you think containers themselves are overkill, or just k8s?
- darren0 8y ago
  Containers have a lot of value. My rule of thumb is basically use containers (docker) and as little as possible of anything else in the container ecosystem. So basically put your app in a container try to not change too much from however you currently do things. There is no shame in running --net=host or just doing "-v /mnt/someebsvolume:/data." I personally like Hashicorp because their tools are simple and grow with you. Besides Hashicorp, I don't have great recommendations for tools. In my opinion the current container ecosystem is a disaster for users. Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture.
  - samstave 8y ago
    Where can i read more about best practice regarding this while not getting to distracted or off in the wrong direction?
    - Serow225 8y ago
      I would also really love to learn the same.
  - antpls 8y ago
    > Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture. I don't work with K8S or even docker on a day-to-day basis, but to me the ecosystem looks rather good. There are several solutions of orchestration and containerization competing from different vendors, and things tend to be standardised to make them work together : - https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md https://github.com/kubernetes/community/blob/master/contribu... - https://www.opencontainers.org/ https://www.opencontainers.org/ No later than yesterday there was that post on HN : https://news.ycombinator.com/item?id=18050781 https://news.ycombinator.com/item?id=18050781 People experiment, dissect, keep some old things, hack new things. K8S is still in active development, new initiatives like Istio and Knative are built on top of it. It looks like an healthy innovative ecosystem to me. I guess it depends of point of view and interests.
- witten 8y ago
  If you just need to "orchestrate" on a single box, Docker Compose (https://docs.docker.com/compose/ https://docs.docker.com/compose/) is the bees knees.
- shareometry 8y ago
  I still don't quite understand why Docker Swarm is so overlooked. It essentially offers easy-to-manage orchestration while providing most of the core features that many people are looking for. It even makes use of compose files, which many developers are already writing. Given that most of the complaints about Kubernetes relate to its complexity and the need for a Kubernetes "expert" to look after said complexity, I'm somewhat baffled by how many people just totally overlook Swarm as if it wasn't an option.
  - ofrzeta 8y ago
    I think you are right but the fact that Docker Inc included Kubernetes surely didn't help Swarm adoption.
  - justinclift 8y ago
    > I still don't quite understand why Docker Swarm is so overlooked. No IPv6 support in it at all, and the company seems to have moved resources internally away from Swarm development. Doesn't bode well for it's future. Expecting it'll be taken out to the back shed (-bang-) in a while.
    - ofrzeta 8y ago
      Check this GitHub issue on SwarmKit for more info: https://github.com/docker/swarmkit/issues/2665 https://github.com/docker/swarmkit/issues/2665
      - justinclift 8y ago
        Docker staff have been saying words to the effect of "we'll make an announcement with info soon!" for at least the last three+ months, across several projects. Swarm, Docker-Machine, & Infrakit come to mind. With no actual announcement(s) then occurring. The commit activity graph on InfraKit - which people are supposed to use now that Docker-Machine has been deprecated - seems pretty clear: https://github.com/docker/infrakit/graphs/contributors https://github.com/docker/infrakit/graphs/contributors SwarmKit is the same: https://github.com/docker/swarmkit/graphs/contributors https://github.com/docker/swarmkit/graphs/contributors Clearly these aren't where resources have been placed. Actions are not lining up with the words given. :(
- matthewmacleod 8y ago
  I have had a good experience using the Hashicorp stack of Nomad and Consul, integrated with nginx for load balancing. This stack doesn't offer all of the same functionality as the Kubernetes ecosystem, but it has the benefit of being vastly simpler and composed of relatively small, independent parts. We did examine Kubernetes, but for various reasons we need to manage our own cluster and the engineering overhead was too high for our team to take on.
  - ngrilly 8y ago
    Why is Nomad & Consul simpler that Kubernetes?
markbnj 8y ago
>> Kubernetes is a big business system, it caters to enterprise. If you aren't an enterprise I would caution you against k8s. That seems overstated, frankly. What is so complex about, for example, GKE that renders it unfit for smaller organizations? If you mean "caution you against hosting your own clusters" I could probably get onboard with that.
- darren0 8y ago
  For sure running k8s is difficult, I don't think anybody would disagree with me there, but of course you can buy GKE. I have to point out that GKE is the only decent k8s implementation, EKS and AKS are not real contenders. Digital Ocean will launch their service, I've seen it and it's good and I hope they do well. But in general you can't really expect the Kubernetes landscape to change much from what it is today, which is basically GKE good, everything else meh. Why? Kubernetes is not really a proven business model for cloud providers yet. It costs the cloud provider more to offer Kubernetes and it not significantly drawing more users in (and consuming more). The obvious example of this is AWS lack luster support and commitment to EKS. If there was a big business opportunity Amazon would be doing more. I have no concrete proof but ECS and Fargate stand to make much more money for Amazon (just a guess). So in general the argument that running on GKE isn't bad I don't think holds up because GKE is the only decent option. DO has yet to launch, but they can't really standup to GKE because you have to look at things holistically. Just running the cluster is like 10% of the whole solution. You need to then figure out how to do authentication, authorization, load balancing (built in service LB and ingress are not sufficient), monitoring, logging, and disaster recovery. Google has a product in each of these areas, but they have 100x more resources than DO. Basically nobody can (or is willing to) put the money into K8s like Google does and Google has yet to prove that their investment is worth it. That's not to say I don't think Kubernetes will survive. I do think it will survive and it will be around for awhile. Because it's a good fit for enterprise right now. Not a good cloud provider business, not good for small dev shops, but good for enterprise.
  - fapjacks 8y ago
    It's interesting you didn't mention Docker once in all of that. I agree with most of what you said, but paying for Docker Enterprise gets you integration with k8s that makes the complexity tolerable and manageable, and also includes all that extra shit that needs to be included. Docker Inc seems to be all-in on this as a core part of their business, so for all the half-hearted ennui of AWS, Docker Inc seems to be heavily invested in solving the problem.
Aeolun 8y ago
Rancher (2.0) certainly made it a lot easier for me as a sole developer to get Kubernetes running in an actually useful way. Though I’m still running into a hundred gotchas a month :P
- ldng 8y ago
  We're still on Rancher 1.6 and Cattle because it's much simpler for us and we don't have the time to invest in a migration to 2.0
  - Aeolun 8y ago
    I would suggest you wait until 2.0 is a bit more mature. It’s fun, but there’s still a lot missing (that appears to be in 1.6)
arountheworld 8y ago
What about Rancher 1.6 and Cattle? Is it going to be improved? It had a perfect balance of complexity and features. I prefer it over k8s.

mwcampbell 8y ago

What do you think most people should use instead of k8s? Or is there not a good answer for that yet? Do you think containers themselves are overkill, or just k8s?

darren0 8y ago
Containers have a lot of value. My rule of thumb is basically use containers (docker) and as little as possible of anything else in the container ecosystem. So basically put your app in a container try to not change too much from however you currently do things. There is no shame in running --net=host or just doing "-v /mnt/someebsvolume:/data." I personally like Hashicorp because their tools are simple and grow with you. Besides Hashicorp, I don't have great recommendations for tools. In my opinion the current container ecosystem is a disaster for users. Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture.
- samstave 8y ago
  Where can i read more about best practice regarding this while not getting to distracted or off in the wrong direction?
  - Serow225 8y ago
    I would also really love to learn the same.
- antpls 8y ago
  > Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture. I don't work with K8S or even docker on a day-to-day basis, but to me the ecosystem looks rather good. There are several solutions of orchestration and containerization competing from different vendors, and things tend to be standardised to make them work together : - https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md https://github.com/kubernetes/community/blob/master/contribu... - https://www.opencontainers.org/ https://www.opencontainers.org/ No later than yesterday there was that post on HN : https://news.ycombinator.com/item?id=18050781 https://news.ycombinator.com/item?id=18050781 People experiment, dissect, keep some old things, hack new things. K8S is still in active development, new initiatives like Istio and Knative are built on top of it. It looks like an healthy innovative ecosystem to me. I guess it depends of point of view and interests.
witten 8y ago
If you just need to "orchestrate" on a single box, Docker Compose (https://docs.docker.com/compose/ https://docs.docker.com/compose/) is the bees knees.
shareometry 8y ago
I still don't quite understand why Docker Swarm is so overlooked. It essentially offers easy-to-manage orchestration while providing most of the core features that many people are looking for. It even makes use of compose files, which many developers are already writing. Given that most of the complaints about Kubernetes relate to its complexity and the need for a Kubernetes "expert" to look after said complexity, I'm somewhat baffled by how many people just totally overlook Swarm as if it wasn't an option.
- ofrzeta 8y ago
  I think you are right but the fact that Docker Inc included Kubernetes surely didn't help Swarm adoption.
- justinclift 8y ago
  > I still don't quite understand why Docker Swarm is so overlooked. No IPv6 support in it at all, and the company seems to have moved resources internally away from Swarm development. Doesn't bode well for it's future. Expecting it'll be taken out to the back shed (-bang-) in a while.
  - ofrzeta 8y ago
    Check this GitHub issue on SwarmKit for more info: https://github.com/docker/swarmkit/issues/2665 https://github.com/docker/swarmkit/issues/2665
    - justinclift 8y ago
      Docker staff have been saying words to the effect of "we'll make an announcement with info soon!" for at least the last three+ months, across several projects. Swarm, Docker-Machine, & Infrakit come to mind. With no actual announcement(s) then occurring. The commit activity graph on InfraKit - which people are supposed to use now that Docker-Machine has been deprecated - seems pretty clear: https://github.com/docker/infrakit/graphs/contributors https://github.com/docker/infrakit/graphs/contributors SwarmKit is the same: https://github.com/docker/swarmkit/graphs/contributors https://github.com/docker/swarmkit/graphs/contributors Clearly these aren't where resources have been placed. Actions are not lining up with the words given. :(
matthewmacleod 8y ago
I have had a good experience using the Hashicorp stack of Nomad and Consul, integrated with nginx for load balancing. This stack doesn't offer all of the same functionality as the Kubernetes ecosystem, but it has the benefit of being vastly simpler and composed of relatively small, independent parts. We did examine Kubernetes, but for various reasons we need to manage our own cluster and the engineering overhead was too high for our team to take on.
- ngrilly 8y ago
  Why is Nomad & Consul simpler that Kubernetes?

markbnj 8y ago

>> Kubernetes is a big business system, it caters to enterprise. If you aren't an enterprise I would caution you against k8s. That seems overstated, frankly. What is so complex about, for example, GKE that renders it unfit for smaller organizations? If you mean "caution you against hosting your own clusters" I could probably get onboard with that.

darren0 8y ago
For sure running k8s is difficult, I don't think anybody would disagree with me there, but of course you can buy GKE. I have to point out that GKE is the only decent k8s implementation, EKS and AKS are not real contenders. Digital Ocean will launch their service, I've seen it and it's good and I hope they do well. But in general you can't really expect the Kubernetes landscape to change much from what it is today, which is basically GKE good, everything else meh. Why? Kubernetes is not really a proven business model for cloud providers yet. It costs the cloud provider more to offer Kubernetes and it not significantly drawing more users in (and consuming more). The obvious example of this is AWS lack luster support and commitment to EKS. If there was a big business opportunity Amazon would be doing more. I have no concrete proof but ECS and Fargate stand to make much more money for Amazon (just a guess). So in general the argument that running on GKE isn't bad I don't think holds up because GKE is the only decent option. DO has yet to launch, but they can't really standup to GKE because you have to look at things holistically. Just running the cluster is like 10% of the whole solution. You need to then figure out how to do authentication, authorization, load balancing (built in service LB and ingress are not sufficient), monitoring, logging, and disaster recovery. Google has a product in each of these areas, but they have 100x more resources than DO. Basically nobody can (or is willing to) put the money into K8s like Google does and Google has yet to prove that their investment is worth it. That's not to say I don't think Kubernetes will survive. I do think it will survive and it will be around for awhile. Because it's a good fit for enterprise right now. Not a good cloud provider business, not good for small dev shops, but good for enterprise.
- fapjacks 8y ago
  It's interesting you didn't mention Docker once in all of that. I agree with most of what you said, but paying for Docker Enterprise gets you integration with k8s that makes the complexity tolerable and manageable, and also includes all that extra shit that needs to be included. Docker Inc seems to be all-in on this as a core part of their business, so for all the half-hearted ennui of AWS, Docker Inc seems to be heavily invested in solving the problem.

Aeolun 8y ago

Rancher (2.0) certainly made it a lot easier for me as a sole developer to get Kubernetes running in an actually useful way. Though I’m still running into a hundred gotchas a month :P

ldng 8y ago
We're still on Rancher 1.6 and Cattle because it's much simpler for us and we don't have the time to invest in a migration to 2.0
- Aeolun 8y ago
  I would suggest you wait until 2.0 is a bit more mature. It’s fun, but there’s still a lot missing (that appears to be in 1.6)

arountheworld 8y ago

What about Rancher 1.6 and Cattle? Is it going to be improved? It had a perfect balance of complexity and features. I prefer it over k8s.