11 ms·

Kubernetes for Developers Who Know How to Develop

115 points by EntICOnc 4y ago

endingwithali 4y ago
author here! Thanks so much for cross posting to HN <3
embik 4y ago
> Kubernetes x Docker: Best Friends Forever > Whenever you hear Kubernetes, you’re probably going to hear Docker in the same sentence. Kubernetes and Docker are like peanut butter and jelly—they’re a perfect pair. For what it's worth, this article was originally written in October 2021, but even at that point it was clear that Kubernetes would remove support for docker. In fact it has already with Kubernetes 1.24. Not exactly "Best Friends Forever". I wish people writing this kind of introductory blog posts would take the time and just quickly explain container runtimes instead of just mentioning docker and be done with it.
- endingwithali 4y ago
  Author here! Yeah, the article is out of date but thats okay - honestly, your suggestion for container runtimes is a great idea - it's now on my list of blog posts to write! I'll be sure to credit you :)
- altdataseller 4y ago
  Does this apply to GKE? Are they going to remove support for Docker as well (or did they?)
  - embik 4y ago
    They have, with 1.24[1], following Kubernetes upstream. If you are an end user nothing is really going to change from this though, because docker produces container images that can be used in other container runtimes (and in fact, containerd is what powers docker underneath today anyway). But that's why it's so important to explain that containers != docker, otherwise "removing docker support" seems really scary. [1] https://cloud.google.com/kubernetes-engine/docs/deprecations/docker-containerd https://cloud.google.com/kubernetes-engine/docs/deprecations...
    - georgyo 4y ago
      > They have The question was did they remove support for docker. They have not. They simply removed dockershim and no longer special case it. Your point is that we should stop using a brand name to describe OCI open standard containers. However, like aspirin, the names are interchangable for the vast majority of people. OCI container doesn't really roll off the toung, and container by itself could mean all sorts of things. Docker is really the best word to describe what people are thinking about it every day conversation. So much so that I think they are at risk of loose their trademark.
      - embik 4y ago
        > The question was did they remove support for docker. They have not. Have they not? Note that the GP asked for GKE specifically. The support page I linked to literally says so: > GKE will stop supporting node images that use Docker as the runtime in GKE version 1.24 and later Removing dockershim removed the existing support for docker, because docker does not support CRI (Container Runtime Interface), the API required by Kubernetes. You can go through a third-party solution that adds CRI support on top of docker, but most managed Kubernetes offerings simply removed docker support. I don't see any argument supporting the claim that docker is the "best word" to describe containers. I am also not aware of ambiguity for the term "(Linux) container" when it comes to operating/deploying software. What else does it mean in that context?
        boloust 4y ago
        Saying "Kubernetes has removed support for Docker" is incredibly misleading at best, and less charitably, is just plainly wrong. While it's true that 1.24 does not support docker as the specific container runtime that's directly used by Kubernetes itself, this has approximately zero impact on how the vast majority of beginners would use Kubernetes, as out of the box you're still able to run docker containers. Probably not the kind of confusing detail that needs to be in an intro to Kubernetes article.
        yjftsjthsd-h 4y ago
- gerwim 4y ago
  While you are technically correct, kubernetes removed the Dockershim. Docker is often used as a synonym for containers. However, most of the things should still work [1]. [1]: https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-...
  - embik 4y ago
    I am very much aware of this, but that's exactly what I am arguing. Articles setting out to explain Kubernetes should not perpetuate the false equality of docker and containers.
    - yonixw 4y ago
      If you use `docker build/run` in your bash, it will actually be more confusing IMO saying k8s "dropped docker(shim) support". This fact should be a footnote for the curious reader, surely on an introduction article.
      - xyzzy_plugh 4y ago
        It would do the industry a world of good to move past conflating Docker with Linux containers, containerization, images, etc. There are tons of ways to build containers without Docker. You're effectively encouraging propagating confusion.
        zekrioca 4y ago
        Indeed, and I have zero idea why people don’t use something like LXC and still prefer Docker. Marketing is hell sometimes.
- stetrain 4y ago
  > remove support for docker So I can't use Dockerfiles and `docker build` to build things for k8s?
  - vbezhenar 4y ago
    This is not correct. Kubernetes used docker engine under the hood in the past. Now they abstracted useful part of this engine into API. There's implementation from the docker (containerd), there's implementation from Redhat (CRI-O), may be others. Docker don't have to be installed for Kubernetes to work anymore. Building container images is a different topic. Kubernetes does not have anything to offer here. So you probably still need docker in your development machine and in your CI pipeline to build those images. There are plenty of alternatives rised in recent years, most prominent ones are kaniko, buildah/podman, but they're far from docker in their maturity. That actually makes a problem. It's hard to run docker and kubernetes side-by-side. Or docker inside kubernetes. So if you want to run your CI jobs inside Kubernetes, there's no good solutions right now. I think people will eventually migrate to Kaniko. It's from Google, it seems to be a sane approach. But right now it's a mess.
    - urthor 4y ago
      This is correct. K8s abstracted out containerization in preparation for the anticipated long term migration of the ecosystem to Podman et al (lots and lots of people are invested in moving "containers" away from "Docker").
      - moondev 4y ago
        I don't think podman is designed or supported as a CRI. It's for local development. You can create local "pods" without an API server, but not running containers on a real Kubernetes node. That would be containerd, crio, or cri-dockerd. With that said I still prefer docker for local development by creating real ephemeral clusters with kind
0x006A 4y ago
What part makes this for developers and what are developers that don't know how to develop?
- birdyrooster 4y ago
  Probably a joke for all those software engineers that don’t do anything.
- politelemon 4y ago
  I've read through twice and I cannot see the relationship between the title and the contents.
  - Izkata 4y ago
    It's the part where it explains the "what" and "why" but skips the "how" under the assumption someone who knows how to develop just needs a hook to relate it to what they already know, and can then learn on their own.
- deleted 4y ago
  [deleted]
ForHackernews 4y ago
Kubernetes was a clever ploy to trick cloud providers into abstracting themselves into irrelevance, but otherwise a dead end. Can we please hurry up and replace it with an open-source Nomad clone?
- vbezhenar 4y ago
  IMO Kubernetes is not going anywhere. People are jumping to it in huge numbers. It's very hot. It has sane architecture in my opinion. I'd say it has awesome architecture and I have great respects to the ones responsible for it. They've got very quality foundations. Kubernetes is new Linux. You expected to know Linux and Git, now you'll be expected to know Kubernetes if you're developer.
  - ok123456 4y ago
    It's not going anywhere in the sense that it's this decade's XML fad. In 20 years people will be whining about their legacy system running Kubernetes, and how it's a nightmare.
  - ForHackernews 4y ago
    Hadoop is not going anywhere. People are jumping to it in huge numbers. It's very hot. It has sane architecture in my opinion. I'd say it has awesome architecture and I have great respects to the ones responsible for it. They've got very quality foundations. Hadoop is new ETL. You expected to know SQL and ETL, now you'll be expected to know Hadoop if you're a developer. ~HN, circa 2010
- yjftsjthsd-h 4y ago
  > replace it with an open-source Nomad clone? Nomad is open source; why clone it?
  - ForHackernews 4y ago
    Is the whole thing open? I assumed it was open-core and the good bits were hasicorp secret sauce.
rschachte 4y ago
I’ve been working on a Kubernetes cluster for my home network. I know a lot of people hate on Kubernetes, but it’s pretty cool and not too bad to even get started from scratch with Kubeadm. I think the constant thought in the back of my mind is… Docker containers and Nginx is definitely enough for my needs. At work, it’s a different story, but more most things, it can be over kill.
- vbezhenar 4y ago
  My biggest issue with Kubernetes is huge costs of running it. It's 3 servers with 4 GB RAM. It's almost $100/month just for Kubernetes alone. And you need one load balancer for control nodes which is another $25 for my hoster. And you need second load balancer for production workloads which is another $25. So you have to pay $150/month for the privilege of using Kubernetes. And they double your costs for your application, because you want it to be high-available, why bother with Kubernetes otherwise. So something that could be run with $5 droplet, suddenly requires $200. They you want to install grafana, ELK, service mesh and whatnot.
  - tux3 4y ago
    k3s will run on a $5 droplet happily. If you want k8s for the yaml and not for the heavy-duty hyperscale redundancy, k3s works great
    - vbezhenar 4y ago
      It's hard to stop when HA is one config line (and few additional dozens of bucks) away.
  - yjftsjthsd-h 4y ago
    You don't actually need all that. If the competition is a single $5 machine with no redundancy, you can just run a single node cluster.
  - jeroenhd 4y ago
    Three machines with 4GB of RAM for $100? I don't know where you rent your servers but that's ridiculous. You can easily get that kind of compute for a third or lower. You won't get a private LAN (so you'll need some firewall rules on the hoster side) and you'll have to do some setup to get everything served up on the right public IP address(es), but you don't need a dedicated load balancer product. Unless you expect your virtual servers to go down randomly, I suppose. I've set up k0s on Oracle's free service to mess with. It doesn't need nearly as much as 12GiB of RAM. Give it 500MiB per machine to be happy and it _should_ run fine with 10.5GiB to spare unless you're adding tons of overlays and additional services on top of your applications. As far as I can tell, the biggest cost with Kubernetes is maintaining the system and keeping up to date with the advancements and deprecations in the Kubernetes field. You can't just upgrade the Kubernetes version and expect everything to work, and every new concept that replaces anything old comes with Kubernetes layers of complexity. I honestly don't know what so many companies are using all those resources Kubernetes provides for. I have a feeling people are so caught up in the "modern server management" world that they've forgotten how powerful and reliable a $25 VPS is these days.
daenz 4y ago
I'm going to post my unsolicited opinion about K8s here, as an SWE+SRE who used it heavily for about 1.5 years on GCP. It's a very cool system. I completely understand why people half-jokingly call it a "distributed operating system." It does a lot of things related to the lifecycles of various state (secrets, storage, config, deployments, etc). However, I believe it goes way too far into putting infrastructure into non-cloud-managed state machines. Things that exist in most modern clouds are being reinvented in K8s. What's more, is that K8s objects are being created as interfaces to the underlying cloud objects. So you now have 2 layers of abstractions, each with their own quirks. It's too much. Not to mention that IaC for K8s is extremely immature. This will improve, yes, for some definition of "improve." But if you've ever written Helm charts that integrate with Terraform, you'll know about all the spinning plates you have to keep balanced. It's not a system I see sustaining into the long term future. Google may continue to use and support it forever, but afaik, they are the most invested in its success. Other cloud platforms, like AWS, seem to be focusing on not re-inventing all of their cloud offerings in K8s.
- endingwithali 4y ago
  author here: interesting - maybe you should also write a blog post about it. would love to learn more about your thoughts in depth!
- cheriot 4y ago
  > Things that exist in most modern clouds are being reinvented in K8s. My more optimistic view is that vendor specific APIs are being standardized. Initial implementations have their issues, but as more people use them the cloud vendors will improve their offering.
  - dogecoinbase 4y ago
    Uniform abstractions are necessarily leaky -- the adoption of the k8s standardization, such as it is, papers over implementation details that serious operators require visibility into.
    - 4y ago
JanMa 4y ago
Why does everyone keep thinking developers need to now about K8S? Just be glad if your infra/DevOps/platform teams abstract this away from you and you don't have to deal with this insanity.
- JacobHenner 4y ago
  Not every org delegates these responsibilities to separate roles.
  - deckard1 4y ago
    if you're at the size of needing K8S, you can afford a devops/cloud engineering team. To OP's point, this seems to be a trend among corporate blogs. Targeting devs with irrelevant content. A few weeks back I remember seeing a "SOC 2 for devs" blog article and was really scratching my head. That's not the level that devs work at, at all. If you're paying six figure salaries to devs to sit around all day worrying about overweight bureaucratic nonsense, or other tasks way beyond their expertise, then you're doing it wrong. I'm guessing these orgs think devs has some decision making influence or perhaps it looks good for recruiting? Or maybe they just want that HN juice. What next, I wonder? Neurosurgery for devs?
    - Tostino 4y ago
      Both of these articles were actually interesting to me as a dev who built a b2b SaaS company.
    - EddySchauHai 4y ago
      What’s SOC 2? Security Operations Center?
      - yjftsjthsd-h 4y ago
        it's an audit https://en.m.wikipedia.org/wiki/System_and_Organization_Controls https://en.m.wikipedia.org/wiki/System_and_Organization_Cont...
hujun 4y ago
the problem I have with k8s or whole concept of "cloud native" is it almost ONLY focus on web based application, if your application is not HTTP based or "stateless", it is very hard to run or design to be "cloud native" , specially in a managed k8s service(like eks/aks).
- slimsag 4y ago
  If you want highly scalable at Google's scale, HTTP-based stateless services that can be scaled up/down in response to traffic are highly desirable properties. The real problem is k8s and "cloud native" are designed for Google's scale, not yours.
  - deleted 4y ago
    [deleted]
  - moondev 4y ago
    The benefit of Kubernetes for most these days is the standardized api, not the ability to "webscale" although it certainly can. Google itself doesn't use k8s internally anyway - it uses borg. The CNCF oversees the Kubernetes project, not Google.
- CameronNemo 4y ago
  Can you tell me what specific challenges you've had with deploying non-HTTP services using Kubernetes? Many features are http-centric, but others seem fairly agnostic to me. What kind of non-http services are we talking about anyway?
  - evol262 4y ago
    Deploying them is easy. Figuring out the correct incantations to bind a load balancer or ingress port to redis/kafka/mysql/whatever and/or expose them in some way other than "let the ingress figure out the host from headers/SNI and map to some thing in the Service" is somewhat less than intuitive. Sure, you could argue that you shouldn't expose backend services like that without an API endpoint at all, because one application->one namespace, or you could use a service mesh/consul to expose it to the things that need it, but it's an entirely different mindset from "I have a postgres cluster in public/private cloud"
vonwoodson 4y ago
k8s is making a lot of the same “write once run anywhere” promises that were made by Jav-hit in the neck with a blow dart-Agggghhhh!…
- yowlingcat 4y ago
  For some reason, I'm hearing this in my head as you saying this right before your back is cracked by a chiropractor. So something along the lines of "Jav-ahhhhhhhhhh."
- steveBK123 4y ago
  billions of devices lol
synergy20 4y ago
I heard serverless/lambda/Faas is to replace microservice/k8s/docker, but the former is 100% tight-coupled to the cloud vendors to me, k8s seems more portable.
- LarsDu 4y ago
  Serverless is just letting the cloud provider do you task using their own managed account under the hood. It creates a very strong vendor lock-in. K8s can be migrated to any cloud or even on-prem with some amount of "ease"
wruza 4y ago
Is there a platform for small companies to ease deployment and maintenance? Something like order vps, apt install that, go to ip:4321, login, enter a master ip, and now it’s part of a farm to which you can drag and drop containers (or repositories, or even just init scripts with tarballs) and edit configs/secrets accordingly, and see logs. I understand docker and what k8s does, but in our situation setting them up and managing is time-consuming overkill that brings nothing except complexity and need for yet another expensive role.
- mgarstecki 4y ago
  You might be best served by some platform-as-a-service (Heroku, platform.sh, Netlify, ...) than by a cluster offering. PaaS are pretty much turnkey for many apps, and cost mostly scales with the actual usage which tends to fit well small companies. They completely hide infra management from you, deployment can be entirely automated through Git with no SSH or Docker registry management, and they provide access to what you listed: config/secrets, logs.
  - wernerb 4y ago
    The interface/dsl should have to be really easy though to make it worth it. E.g., if you are making your own buildpack on heroku you have gone too far. The whole idea should be ubiquity in knowledge of these platforms. The idea that you can easily hire someone or expect someone to have these skills. Kubernetes might not be simple but it is becoming ubiquitous and that is worth something in the long run. So if you are looking for the easy solution, use a SaaS / PaaS that really is turnkey and, for example is not just another peaky abstraction of kubernetes. You are choosing an easy solution so you should not need to invest a lot of your energy. Likewise recognize that when you need more customization that you might need to jump ship instead of investing in ducktaping your SaaS/PaaS.
  - wruza 4y ago
    I think we are already using paas (DO Apps) for a few services, but some requirements are incompatible with it and other paas’es. I’d like to rather have a thin layer over vps, what I do with ssh, but better UX.

115 points by EntICOnc 4y ago

author here! Thanks so much for cross posting to HN <3

> Kubernetes x Docker: Best Friends Forever > Whenever you hear Kubernetes, you’re probably going to hear Docker in the same sentence. Kubernetes and Docker are like peanut butter and jelly—they’re a perfect pair. For what it's worth, this article was originally written in October 2021, but even at that point it was clear that Kubernetes would remove support for docker. In fact it has already with Kubernetes 1.24. Not exactly "Best Friends Forever". I wish people writing this kind of introductory blog posts would take the time and just quickly explain container runtimes instead of just mentioning docker and be done with it.

endingwithali 4y ago
Author here! Yeah, the article is out of date but thats okay - honestly, your suggestion for container runtimes is a great idea - it's now on my list of blog posts to write! I'll be sure to credit you :)
altdataseller 4y ago
Does this apply to GKE? Are they going to remove support for Docker as well (or did they?)
- embik 4y ago
  They have, with 1.24[1], following Kubernetes upstream. If you are an end user nothing is really going to change from this though, because docker produces container images that can be used in other container runtimes (and in fact, containerd is what powers docker underneath today anyway). But that's why it's so important to explain that containers != docker, otherwise "removing docker support" seems really scary. [1] https://cloud.google.com/kubernetes-engine/docs/deprecations/docker-containerd https://cloud.google.com/kubernetes-engine/docs/deprecations...
  - georgyo 4y ago
    > They have The question was did they remove support for docker. They have not. They simply removed dockershim and no longer special case it. Your point is that we should stop using a brand name to describe OCI open standard containers. However, like aspirin, the names are interchangable for the vast majority of people. OCI container doesn't really roll off the toung, and container by itself could mean all sorts of things. Docker is really the best word to describe what people are thinking about it every day conversation. So much so that I think they are at risk of loose their trademark.
    - embik 4y ago
      > The question was did they remove support for docker. They have not. Have they not? Note that the GP asked for GKE specifically. The support page I linked to literally says so: > GKE will stop supporting node images that use Docker as the runtime in GKE version 1.24 and later Removing dockershim removed the existing support for docker, because docker does not support CRI (Container Runtime Interface), the API required by Kubernetes. You can go through a third-party solution that adds CRI support on top of docker, but most managed Kubernetes offerings simply removed docker support. I don't see any argument supporting the claim that docker is the "best word" to describe containers. I am also not aware of ambiguity for the term "(Linux) container" when it comes to operating/deploying software. What else does it mean in that context?
      - boloust 4y ago
        Saying "Kubernetes has removed support for Docker" is incredibly misleading at best, and less charitably, is just plainly wrong. While it's true that 1.24 does not support docker as the specific container runtime that's directly used by Kubernetes itself, this has approximately zero impact on how the vast majority of beginners would use Kubernetes, as out of the box you're still able to run docker containers. Probably not the kind of confusing detail that needs to be in an intro to Kubernetes article.
        yjftsjthsd-h 4y ago
gerwim 4y ago
While you are technically correct, kubernetes removed the Dockershim. Docker is often used as a synonym for containers. However, most of the things should still work [1]. [1]: https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/ https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-...
- embik 4y ago
  I am very much aware of this, but that's exactly what I am arguing. Articles setting out to explain Kubernetes should not perpetuate the false equality of docker and containers.
  - yonixw 4y ago
    If you use `docker build/run` in your bash, it will actually be more confusing IMO saying k8s "dropped docker(shim) support". This fact should be a footnote for the curious reader, surely on an introduction article.
    - xyzzy_plugh 4y ago
      It would do the industry a world of good to move past conflating Docker with Linux containers, containerization, images, etc. There are tons of ways to build containers without Docker. You're effectively encouraging propagating confusion.
      - zekrioca 4y ago
        Indeed, and I have zero idea why people don’t use something like LXC and still prefer Docker. Marketing is hell sometimes.
stetrain 4y ago
> remove support for docker So I can't use Dockerfiles and `docker build` to build things for k8s?
- vbezhenar 4y ago
  This is not correct. Kubernetes used docker engine under the hood in the past. Now they abstracted useful part of this engine into API. There's implementation from the docker (containerd), there's implementation from Redhat (CRI-O), may be others. Docker don't have to be installed for Kubernetes to work anymore. Building container images is a different topic. Kubernetes does not have anything to offer here. So you probably still need docker in your development machine and in your CI pipeline to build those images. There are plenty of alternatives rised in recent years, most prominent ones are kaniko, buildah/podman, but they're far from docker in their maturity. That actually makes a problem. It's hard to run docker and kubernetes side-by-side. Or docker inside kubernetes. So if you want to run your CI jobs inside Kubernetes, there's no good solutions right now. I think people will eventually migrate to Kaniko. It's from Google, it seems to be a sane approach. But right now it's a mess.
  - urthor 4y ago
    This is correct. K8s abstracted out containerization in preparation for the anticipated long term migration of the ecosystem to Podman et al (lots and lots of people are invested in moving "containers" away from "Docker").
    - moondev 4y ago
      I don't think podman is designed or supported as a CRI. It's for local development. You can create local "pods" without an API server, but not running containers on a real Kubernetes node. That would be containerd, crio, or cri-dockerd. With that said I still prefer docker for local development by creating real ephemeral clusters with kind

0x006A 4y ago

What part makes this for developers and what are developers that don't know how to develop?

birdyrooster 4y ago
Probably a joke for all those software engineers that don’t do anything.
politelemon 4y ago
I've read through twice and I cannot see the relationship between the title and the contents.
- Izkata 4y ago
  It's the part where it explains the "what" and "why" but skips the "how" under the assumption someone who knows how to develop just needs a hook to relate it to what they already know, and can then learn on their own.
deleted 4y ago
[deleted]

ForHackernews 4y ago

Kubernetes was a clever ploy to trick cloud providers into abstracting themselves into irrelevance, but otherwise a dead end. Can we please hurry up and replace it with an open-source Nomad clone?

vbezhenar 4y ago
IMO Kubernetes is not going anywhere. People are jumping to it in huge numbers. It's very hot. It has sane architecture in my opinion. I'd say it has awesome architecture and I have great respects to the ones responsible for it. They've got very quality foundations. Kubernetes is new Linux. You expected to know Linux and Git, now you'll be expected to know Kubernetes if you're developer.
- ok123456 4y ago
  It's not going anywhere in the sense that it's this decade's XML fad. In 20 years people will be whining about their legacy system running Kubernetes, and how it's a nightmare.
- ForHackernews 4y ago
  Hadoop is not going anywhere. People are jumping to it in huge numbers. It's very hot. It has sane architecture in my opinion. I'd say it has awesome architecture and I have great respects to the ones responsible for it. They've got very quality foundations. Hadoop is new ETL. You expected to know SQL and ETL, now you'll be expected to know Hadoop if you're a developer. ~HN, circa 2010
yjftsjthsd-h 4y ago
> replace it with an open-source Nomad clone? Nomad is open source; why clone it?
- ForHackernews 4y ago
  Is the whole thing open? I assumed it was open-core and the good bits were hasicorp secret sauce.

rschachte 4y ago

I’ve been working on a Kubernetes cluster for my home network. I know a lot of people hate on Kubernetes, but it’s pretty cool and not too bad to even get started from scratch with Kubeadm. I think the constant thought in the back of my mind is… Docker containers and Nginx is definitely enough for my needs. At work, it’s a different story, but more most things, it can be over kill.

vbezhenar 4y ago
My biggest issue with Kubernetes is huge costs of running it. It's 3 servers with 4 GB RAM. It's almost $100/month just for Kubernetes alone. And you need one load balancer for control nodes which is another $25 for my hoster. And you need second load balancer for production workloads which is another $25. So you have to pay $150/month for the privilege of using Kubernetes. And they double your costs for your application, because you want it to be high-available, why bother with Kubernetes otherwise. So something that could be run with $5 droplet, suddenly requires $200. They you want to install grafana, ELK, service mesh and whatnot.
- tux3 4y ago
  k3s will run on a $5 droplet happily. If you want k8s for the yaml and not for the heavy-duty hyperscale redundancy, k3s works great
  - vbezhenar 4y ago
    It's hard to stop when HA is one config line (and few additional dozens of bucks) away.
- yjftsjthsd-h 4y ago
  You don't actually need all that. If the competition is a single $5 machine with no redundancy, you can just run a single node cluster.
- jeroenhd 4y ago
  Three machines with 4GB of RAM for $100? I don't know where you rent your servers but that's ridiculous. You can easily get that kind of compute for a third or lower. You won't get a private LAN (so you'll need some firewall rules on the hoster side) and you'll have to do some setup to get everything served up on the right public IP address(es), but you don't need a dedicated load balancer product. Unless you expect your virtual servers to go down randomly, I suppose. I've set up k0s on Oracle's free service to mess with. It doesn't need nearly as much as 12GiB of RAM. Give it 500MiB per machine to be happy and it _should_ run fine with 10.5GiB to spare unless you're adding tons of overlays and additional services on top of your applications. As far as I can tell, the biggest cost with Kubernetes is maintaining the system and keeping up to date with the advancements and deprecations in the Kubernetes field. You can't just upgrade the Kubernetes version and expect everything to work, and every new concept that replaces anything old comes with Kubernetes layers of complexity. I honestly don't know what so many companies are using all those resources Kubernetes provides for. I have a feeling people are so caught up in the "modern server management" world that they've forgotten how powerful and reliable a $25 VPS is these days.

daenz 4y ago

I'm going to post my unsolicited opinion about K8s here, as an SWE+SRE who used it heavily for about 1.5 years on GCP. It's a very cool system. I completely understand why people half-jokingly call it a "distributed operating system." It does a lot of things related to the lifecycles of various state (secrets, storage, config, deployments, etc). However, I believe it goes way too far into putting infrastructure into non-cloud-managed state machines. Things that exist in most modern clouds are being reinvented in K8s. What's more, is that K8s objects are being created as interfaces to the underlying cloud objects. So you now have 2 layers of abstractions, each with their own quirks. It's too much. Not to mention that IaC for K8s is extremely immature. This will improve, yes, for some definition of "improve." But if you've ever written Helm charts that integrate with Terraform, you'll know about all the spinning plates you have to keep balanced. It's not a system I see sustaining into the long term future. Google may continue to use and support it forever, but afaik, they are the most invested in its success. Other cloud platforms, like AWS, seem to be focusing on not re-inventing all of their cloud offerings in K8s.

endingwithali 4y ago
author here: interesting - maybe you should also write a blog post about it. would love to learn more about your thoughts in depth!
cheriot 4y ago
> Things that exist in most modern clouds are being reinvented in K8s. My more optimistic view is that vendor specific APIs are being standardized. Initial implementations have their issues, but as more people use them the cloud vendors will improve their offering.
- dogecoinbase 4y ago
  Uniform abstractions are necessarily leaky -- the adoption of the k8s standardization, such as it is, papers over implementation details that serious operators require visibility into.
  - 4y ago

JanMa 4y ago

Why does everyone keep thinking developers need to now about K8S? Just be glad if your infra/DevOps/platform teams abstract this away from you and you don't have to deal with this insanity.

JacobHenner 4y ago
Not every org delegates these responsibilities to separate roles.
- deckard1 4y ago
  if you're at the size of needing K8S, you can afford a devops/cloud engineering team. To OP's point, this seems to be a trend among corporate blogs. Targeting devs with irrelevant content. A few weeks back I remember seeing a "SOC 2 for devs" blog article and was really scratching my head. That's not the level that devs work at, at all. If you're paying six figure salaries to devs to sit around all day worrying about overweight bureaucratic nonsense, or other tasks way beyond their expertise, then you're doing it wrong. I'm guessing these orgs think devs has some decision making influence or perhaps it looks good for recruiting? Or maybe they just want that HN juice. What next, I wonder? Neurosurgery for devs?
  - Tostino 4y ago
    Both of these articles were actually interesting to me as a dev who built a b2b SaaS company.
  - EddySchauHai 4y ago
    What’s SOC 2? Security Operations Center?
    - yjftsjthsd-h 4y ago
      it's an audit https://en.m.wikipedia.org/wiki/System_and_Organization_Controls https://en.m.wikipedia.org/wiki/System_and_Organization_Cont...

hujun 4y ago

the problem I have with k8s or whole concept of "cloud native" is it almost ONLY focus on web based application, if your application is not HTTP based or "stateless", it is very hard to run or design to be "cloud native" , specially in a managed k8s service(like eks/aks).

slimsag 4y ago
If you want highly scalable at Google's scale, HTTP-based stateless services that can be scaled up/down in response to traffic are highly desirable properties. The real problem is k8s and "cloud native" are designed for Google's scale, not yours.
- deleted 4y ago
  [deleted]
- moondev 4y ago
  The benefit of Kubernetes for most these days is the standardized api, not the ability to "webscale" although it certainly can. Google itself doesn't use k8s internally anyway - it uses borg. The CNCF oversees the Kubernetes project, not Google.
CameronNemo 4y ago
Can you tell me what specific challenges you've had with deploying non-HTTP services using Kubernetes? Many features are http-centric, but others seem fairly agnostic to me. What kind of non-http services are we talking about anyway?
- evol262 4y ago
  Deploying them is easy. Figuring out the correct incantations to bind a load balancer or ingress port to redis/kafka/mysql/whatever and/or expose them in some way other than "let the ingress figure out the host from headers/SNI and map to some thing in the Service" is somewhat less than intuitive. Sure, you could argue that you shouldn't expose backend services like that without an API endpoint at all, because one application->one namespace, or you could use a service mesh/consul to expose it to the things that need it, but it's an entirely different mindset from "I have a postgres cluster in public/private cloud"

vonwoodson 4y ago

k8s is making a lot of the same “write once run anywhere” promises that were made by Jav-hit in the neck with a blow dart-Agggghhhh!…

yowlingcat 4y ago
For some reason, I'm hearing this in my head as you saying this right before your back is cracked by a chiropractor. So something along the lines of "Jav-ahhhhhhhhhh."
steveBK123 4y ago
billions of devices lol

synergy20 4y ago

I heard serverless/lambda/Faas is to replace microservice/k8s/docker, but the former is 100% tight-coupled to the cloud vendors to me, k8s seems more portable.

LarsDu 4y ago
Serverless is just letting the cloud provider do you task using their own managed account under the hood. It creates a very strong vendor lock-in. K8s can be migrated to any cloud or even on-prem with some amount of "ease"

wruza 4y ago

Is there a platform for small companies to ease deployment and maintenance? Something like order vps, apt install that, go to ip:4321, login, enter a master ip, and now it’s part of a farm to which you can drag and drop containers (or repositories, or even just init scripts with tarballs) and edit configs/secrets accordingly, and see logs. I understand docker and what k8s does, but in our situation setting them up and managing is time-consuming overkill that brings nothing except complexity and need for yet another expensive role.

mgarstecki 4y ago
You might be best served by some platform-as-a-service (Heroku, platform.sh, Netlify, ...) than by a cluster offering. PaaS are pretty much turnkey for many apps, and cost mostly scales with the actual usage which tends to fit well small companies. They completely hide infra management from you, deployment can be entirely automated through Git with no SSH or Docker registry management, and they provide access to what you listed: config/secrets, logs.
- wernerb 4y ago
  The interface/dsl should have to be really easy though to make it worth it. E.g., if you are making your own buildpack on heroku you have gone too far. The whole idea should be ubiquity in knowledge of these platforms. The idea that you can easily hire someone or expect someone to have these skills. Kubernetes might not be simple but it is becoming ubiquitous and that is worth something in the long run. So if you are looking for the easy solution, use a SaaS / PaaS that really is turnkey and, for example is not just another peaky abstraction of kubernetes. You are choosing an easy solution so you should not need to invest a lot of your energy. Likewise recognize that when you need more customization that you might need to jump ship instead of investing in ducktaping your SaaS/PaaS.
- wruza 4y ago
  I think we are already using paas (DO Apps) for a few services, but some requirements are incompatible with it and other paas’es. I’d like to rather have a thin layer over vps, what I do with ssh, but better UX.