6 ms·

So instead of knowing about K8s services, ingests and deployments/pods I have to learn 15 tools. Ingests are not much more complicated than an nginx config, se

So instead of knowing about K8s services, ingests and deployments/pods I have to learn 15 tools. Ingests are not much more complicated than an nginx config, services are literally 5 lines each pod, and the deployments are roughly as complicated as a 15 line docker file.

smoldesu 5y ago
If you're familiar with Linux (which should be considered required-reading if you're learning about containers), most of this stuff is handled perfectly fine by the operating system. Sure, you could write it all in K8 and just let the layers of abstraction pile up. Or, most people will be suited perfectly fine by the software that already runs in their box.
- echelon 5y ago
  > most of this stuff is handled perfectly fine by the operating system No, you have to write or adopt tools for each of these things. They don't just magically happen. Then you have to maintain, secure, integrate. k8s solves a broad class of problems in an elegant way. Since other people have adopted it, it gets patched and improved. And you can easily hire for the skillset.
- emerongi 5y ago
  I work in a small company, we don't have a sysadmin, so mostly we want to use managed services. Let's say we want a simple load balanced setup with 2 nodes. Our options are: - Run our own load balancing machine and manage it (as said, we don't want this) - Use AWS/GCP/Azure, setup Load Balancer (and rest of the project) manually or with Terraform/CloudFormation/whatever scripts - Use AWS/GCP/Azure and Kubernetes, define Load Balancer in YAML, let K8S and the platform handle all the boring stuff This is the simplest setup and already I will always go for Kubernetes, as it's the fastest and simplest, as well as the most easily maintainable. I can also easily slap on new services, upgrade stuff, etc. Being able to define the whole architecture in a declarative way, without actually having to manually do the changes, is a huge time-saver. Especially in our case, where we have more projects than developers - switching context from one project to another is much easier. Not to mention that I can just start a development environment with all the needed services using the same (or very similar) manifests, creating a near-prod environment.
  - philjohn 5y ago
    I think the argument there is that it's only simple because the complexity of k8s has been taken away. I don't think anybody has claimed deploying to a k8s cluster is overly complex; running it well, handling upgrades, those are huge time sinks that need the requisite expertise. Much like Multics was "simple" for the users, but not for the sysadmins.
    - raffraffraff 5y ago
      Taking the complexity of k8s away was just gonna happen. As someone who built everything from scratch at a previous company, I chose eks at a start-up because it meant that the one-man-systemsguy didn't have to worry about building and hosting every single cog wheel that is required for package repos, OS deployment, configuration management, consul+vault (minimum), and too many other things that k8s does for you. Also, you can send someone on a CKA course and they know how your shit works. Try doing that with the hodge-podge system you built.
      - ethbr0 5y ago
        Training is a great point, and I think that's why major clouds are going to be stickiest (in terms of using them vs migrating to new things). The central problem of most companies has been finding / affording people who can maintain their stuff. If Amazon / MS / Google can make it simple enough that skilled people can be quickly cross trained, and then have enough architecture knowledge to be productive, that's a huge win over "require everyone to spend 6 months muddling through and learning our stack we built ourselves and partially documented."
    - thinkharderdev 5y ago
      That's the point though right? A good (couple of) sysadmins can run a k8s cluster that can be leveraged by dozens (even hundreds) of dev teams. Instead of every team having to re-invent the wheel you get a common platform and set of deployment patterns that can fit most any use case. Of course if you don't have multiple different teams (or every team is running their own k8s cluster) then that is definitely a problem. But just because a handful of teams make an ill-advised investment in k8s when they could do easily with something much simpler doesn't mean that k8s is "too complex." Too complex for that use case sure, but for the vast majority of k8s deployments I would wager that it does add a lot of value and subsume a lot of the inherent complexity of running distributed, fault-tolerant, multi-tenant applications.
- ownagefool 5y ago
  Okay, so let's add a couple of things. How do you do failover? Sharing servers to save on costs? Orchestrate CI/CD pipelines, preferably on the fly? Infrastructure as Code? Eventually you a point where the abstraction wins. Most people will say "but AWS...", but the reality is quicker, easier to use, and runs via multiple providers, so I think it's going to keep doing well personally.
  - tharkun__ 5y ago
    Not the OP here. We aren't really comparing apples and oranges in all cases that have been talked about in the larger thread. Some of the comparisons seem to be between "self hosted LAMP stack" vs. "kubernetes as a service on AWS". These are vastly different things. We should compare "self hosted LAMP stack" vs. "hosted in cloud LAMP stack" for example or "self-hosted kubernetes" vs. "self-administered kubernetes on EC2" vs "kubernetes as a service on AWS". All of these will have vastly different characteristics, pros and cons depending on your company and teams' realities. Failover is something that a load balancer does automatically for you. Your services just need to provide a health check. Now where you actually run those nodes is a different thing. These might be slow to procure servers hosted at your provider. Or these might be manually set up EC2 instances or terraformed EC2 instances. Dunno what everyone uses as load balancers nowadays but a previous place for example had F5s and we had our own vsphere farm. Sharing servers: I don't think this is a good idea at all except if you mean internally and if you do that then there's good and bad ways (see above on vsphere farm. If one project caused another to starve performance wise because of what was running on the same physical machines it was easy to resolve. If this was virtual servers at a traditional hoster, good luck. AWS is probably somewhere in between with EC2 and especially their storage. Dedicated CI/CD pipelines: This is an awesome one to have and can cost an arm and a leg. I enjoy this very much at my current place w/ EC2 CI agents that scale with the number of devs currently working and dedicated "complete copy of Prod" dev environments (basically a kubernetes namespace for each dev/QA person/e2e test run to play with as they like). Infra as code: Does not require kubernetes at all but can be implemented with kubernetes. If you already used docker to run stuff anyway for example and you can "abstract away" the kubernetes complexities to your SRE team and/or AWS, go ahead and use kubernetes. But be aware that if nobody at your place actually know kubernetes because you just relied on the hosted version of it, you're at the whim of their support people when something blows up in Production. You may not be big enough to have your own SRE team to take care of this but then you might also just not really benefit enough from kubernetes complexity and a simpler arrangement could have been easier for the people you do have to actually understand.
- tapoxi 5y ago
  This really depends on how many boxes you have.
- eropple 5y ago
  I have been professionally working in the infrastructure space for a decade and in an amateur fashion running Linux servers and services for another decade before that and I am pretty certain that I would screw this up in a threat-to-production way at least once or twice along the way and possibly hit a failure-to-launch on the product itself. I would then have to wrestle with the cognitive load of All That Stuff and by the way? The failure case, from a security perspective, of a moment's inattention has unbounded consequences. (The failure case from a scaling perspective is less so! But still bad.) And I mean, I don't even like k8s. I typically go for the AWS suite of stuff when building out systems infrastructure. But this assertion is bonkers.
  - Aeolun 5y ago
    Why? You still need to manage all that for your server even if you are running kubernetes on top of it. I can’t imagine anyone with root access to a kubernetes server is any less dangerous that root on a simple webserver.
    - eropple 5y ago
      No, I don't, because I can yawn dramatically and I can go to any cloud provider and get a k8s cluster with generally consistent and at worst a moral-equivalent set of standard building-block cloud tools already set up. It won't cost me much, it will work mostly-predictably out of the box, and there's support right there for when it fails. Like, that's what k8s is there for. I use AWS pretty exclusively so this doesn't appeal to me, but what does is doing the moral equivalent and having ECS just...there. (Or even better, Fargate, if I can't solve the bin packing problem by myself.) I haven't "managed a server" outside of my house for a few years now, and I quite like it. I theoretically have had root to ECS clusters, but I've never logged into them. Why would I? Amazon is going to be better at it than I am. Not only do I have more important things to be doing, but I'll do a worse job of it than they will. And to be clear: I consider myself pretty kinda really good at this stuff. But not good enough to make it a competitive advantage unless it's what I want to sell, and I sure as heck don't. And the article's point, that whatever comes next will probably be better and might even be The Real Thing--I think that is wise.
cat199 5y ago
> I have to learn 15 tools. kubectl, kustomize, helm, istio, graphana, various flavors of ingress controllers, overlay networks, service meshes, storage controllers, etcd, etc. from tfa: https://landscape.cncf.io/ https://landscape.cncf.io/ you still have to learn 15 tools, just now they are hidden behind the scenes, and you still have to understand the underlying systems to reason about your containers. this isn't for or against k8s - i'm a right tool for the job guy - but as a tool kubernetes doesn't solve problems, it encapsulates them and shifts them around.
- nikau 5y ago
  Plus all the cloud tools are immature with terrible error handling and logging. So after an enjoyable time crafting a 30 level deep json file you get a failed helm deployment with a error message like "timed out waiting for the condition".
nikau 5y ago
15 mature well documented tools are a lot easier than 15 kludged ill thought out Kubernetes definitions. Any serious Kubernetes environment is not 5 lines per pod, its the hell of rbac and pod security policies and all sorts of overly cryptic cruft.

smoldesu 5y ago

If you're familiar with Linux (which should be considered required-reading if you're learning about containers), most of this stuff is handled perfectly fine by the operating system. Sure, you could write it all in K8 and just let the layers of abstraction pile up. Or, most people will be suited perfectly fine by the software that already runs in their box.

echelon 5y ago
> most of this stuff is handled perfectly fine by the operating system No, you have to write or adopt tools for each of these things. They don't just magically happen. Then you have to maintain, secure, integrate. k8s solves a broad class of problems in an elegant way. Since other people have adopted it, it gets patched and improved. And you can easily hire for the skillset.
emerongi 5y ago
I work in a small company, we don't have a sysadmin, so mostly we want to use managed services. Let's say we want a simple load balanced setup with 2 nodes. Our options are: - Run our own load balancing machine and manage it (as said, we don't want this) - Use AWS/GCP/Azure, setup Load Balancer (and rest of the project) manually or with Terraform/CloudFormation/whatever scripts - Use AWS/GCP/Azure and Kubernetes, define Load Balancer in YAML, let K8S and the platform handle all the boring stuff This is the simplest setup and already I will always go for Kubernetes, as it's the fastest and simplest, as well as the most easily maintainable. I can also easily slap on new services, upgrade stuff, etc. Being able to define the whole architecture in a declarative way, without actually having to manually do the changes, is a huge time-saver. Especially in our case, where we have more projects than developers - switching context from one project to another is much easier. Not to mention that I can just start a development environment with all the needed services using the same (or very similar) manifests, creating a near-prod environment.
- philjohn 5y ago
  I think the argument there is that it's only simple because the complexity of k8s has been taken away. I don't think anybody has claimed deploying to a k8s cluster is overly complex; running it well, handling upgrades, those are huge time sinks that need the requisite expertise. Much like Multics was "simple" for the users, but not for the sysadmins.
  - raffraffraff 5y ago
    Taking the complexity of k8s away was just gonna happen. As someone who built everything from scratch at a previous company, I chose eks at a start-up because it meant that the one-man-systemsguy didn't have to worry about building and hosting every single cog wheel that is required for package repos, OS deployment, configuration management, consul+vault (minimum), and too many other things that k8s does for you. Also, you can send someone on a CKA course and they know how your shit works. Try doing that with the hodge-podge system you built.
    - ethbr0 5y ago
      Training is a great point, and I think that's why major clouds are going to be stickiest (in terms of using them vs migrating to new things). The central problem of most companies has been finding / affording people who can maintain their stuff. If Amazon / MS / Google can make it simple enough that skilled people can be quickly cross trained, and then have enough architecture knowledge to be productive, that's a huge win over "require everyone to spend 6 months muddling through and learning our stack we built ourselves and partially documented."
  - thinkharderdev 5y ago
    That's the point though right? A good (couple of) sysadmins can run a k8s cluster that can be leveraged by dozens (even hundreds) of dev teams. Instead of every team having to re-invent the wheel you get a common platform and set of deployment patterns that can fit most any use case. Of course if you don't have multiple different teams (or every team is running their own k8s cluster) then that is definitely a problem. But just because a handful of teams make an ill-advised investment in k8s when they could do easily with something much simpler doesn't mean that k8s is "too complex." Too complex for that use case sure, but for the vast majority of k8s deployments I would wager that it does add a lot of value and subsume a lot of the inherent complexity of running distributed, fault-tolerant, multi-tenant applications.
ownagefool 5y ago
Okay, so let's add a couple of things. How do you do failover? Sharing servers to save on costs? Orchestrate CI/CD pipelines, preferably on the fly? Infrastructure as Code? Eventually you a point where the abstraction wins. Most people will say "but AWS...", but the reality is quicker, easier to use, and runs via multiple providers, so I think it's going to keep doing well personally.
- tharkun__ 5y ago
  Not the OP here. We aren't really comparing apples and oranges in all cases that have been talked about in the larger thread. Some of the comparisons seem to be between "self hosted LAMP stack" vs. "kubernetes as a service on AWS". These are vastly different things. We should compare "self hosted LAMP stack" vs. "hosted in cloud LAMP stack" for example or "self-hosted kubernetes" vs. "self-administered kubernetes on EC2" vs "kubernetes as a service on AWS". All of these will have vastly different characteristics, pros and cons depending on your company and teams' realities. Failover is something that a load balancer does automatically for you. Your services just need to provide a health check. Now where you actually run those nodes is a different thing. These might be slow to procure servers hosted at your provider. Or these might be manually set up EC2 instances or terraformed EC2 instances. Dunno what everyone uses as load balancers nowadays but a previous place for example had F5s and we had our own vsphere farm. Sharing servers: I don't think this is a good idea at all except if you mean internally and if you do that then there's good and bad ways (see above on vsphere farm. If one project caused another to starve performance wise because of what was running on the same physical machines it was easy to resolve. If this was virtual servers at a traditional hoster, good luck. AWS is probably somewhere in between with EC2 and especially their storage. Dedicated CI/CD pipelines: This is an awesome one to have and can cost an arm and a leg. I enjoy this very much at my current place w/ EC2 CI agents that scale with the number of devs currently working and dedicated "complete copy of Prod" dev environments (basically a kubernetes namespace for each dev/QA person/e2e test run to play with as they like). Infra as code: Does not require kubernetes at all but can be implemented with kubernetes. If you already used docker to run stuff anyway for example and you can "abstract away" the kubernetes complexities to your SRE team and/or AWS, go ahead and use kubernetes. But be aware that if nobody at your place actually know kubernetes because you just relied on the hosted version of it, you're at the whim of their support people when something blows up in Production. You may not be big enough to have your own SRE team to take care of this but then you might also just not really benefit enough from kubernetes complexity and a simpler arrangement could have been easier for the people you do have to actually understand.
tapoxi 5y ago
This really depends on how many boxes you have.
eropple 5y ago
I have been professionally working in the infrastructure space for a decade and in an amateur fashion running Linux servers and services for another decade before that and I am pretty certain that I would screw this up in a threat-to-production way at least once or twice along the way and possibly hit a failure-to-launch on the product itself. I would then have to wrestle with the cognitive load of All That Stuff and by the way? The failure case, from a security perspective, of a moment's inattention has unbounded consequences. (The failure case from a scaling perspective is less so! But still bad.) And I mean, I don't even like k8s. I typically go for the AWS suite of stuff when building out systems infrastructure. But this assertion is bonkers.
- Aeolun 5y ago
  Why? You still need to manage all that for your server even if you are running kubernetes on top of it. I can’t imagine anyone with root access to a kubernetes server is any less dangerous that root on a simple webserver.
  - eropple 5y ago
    No, I don't, because I can yawn dramatically and I can go to any cloud provider and get a k8s cluster with generally consistent and at worst a moral-equivalent set of standard building-block cloud tools already set up. It won't cost me much, it will work mostly-predictably out of the box, and there's support right there for when it fails. Like, that's what k8s is there for. I use AWS pretty exclusively so this doesn't appeal to me, but what does is doing the moral equivalent and having ECS just...there. (Or even better, Fargate, if I can't solve the bin packing problem by myself.) I haven't "managed a server" outside of my house for a few years now, and I quite like it. I theoretically have had root to ECS clusters, but I've never logged into them. Why would I? Amazon is going to be better at it than I am. Not only do I have more important things to be doing, but I'll do a worse job of it than they will. And to be clear: I consider myself pretty kinda really good at this stuff. But not good enough to make it a competitive advantage unless it's what I want to sell, and I sure as heck don't. And the article's point, that whatever comes next will probably be better and might even be The Real Thing--I think that is wise.

cat199 5y ago

> I have to learn 15 tools. kubectl, kustomize, helm, istio, graphana, various flavors of ingress controllers, overlay networks, service meshes, storage controllers, etcd, etc. from tfa: https://landscape.cncf.io/ https://landscape.cncf.io/ you still have to learn 15 tools, just now they are hidden behind the scenes, and you still have to understand the underlying systems to reason about your containers. this isn't for or against k8s - i'm a right tool for the job guy - but as a tool kubernetes doesn't solve problems, it encapsulates them and shifts them around.

nikau 5y ago
Plus all the cloud tools are immature with terrible error handling and logging. So after an enjoyable time crafting a 30 level deep json file you get a failed helm deployment with a error message like "timed out waiting for the condition".

nikau 5y ago

15 mature well documented tools are a lot easier than 15 kludged ill thought out Kubernetes definitions. Any serious Kubernetes environment is not 5 lines per pod, its the hell of rbac and pod security policies and all sorts of overly cryptic cruft.