6 ms·

Hi danpalmer, I kind of hit on this in the article. How do you think abstracting away the server adds any value in "public cloud"? Where you can get bespoke VMs

Hi danpalmer, I kind of hit on this in the article. How do you think abstracting away the server adds any value in "public cloud"? Where you can get bespoke VMs, with no concern for the underlying hardware? Can you elaborate?

charcircuit 5y ago
I think part of the confusion is that server can refer to both a physical box and a VM. Abstracting away servers is about not having to think about VMs. You have a service which requires some amount of compute, memory, and storage and you just want that service to run. There is value to not having to worry about provisioning a VM or administering it.
- jasonrichardsmi 5y ago
  But you still have to do that in Kubernetes, unless you are running Fargate. Someone has to provision and maintain that machine, and in the process introduce a ton of administrative overhead.
  - deleted 5y ago
    [deleted]
  - deleted 5y ago
    [deleted]
  - danpalmer 5y ago
    That's true, but not a particularly interesting fact. For example, if you use GKE (Google Cloud's K8s offering), you attach your K8s cluster to an auto-scaling node pool and it handles (de)provisioning of your VMs for you. You essentially don't care about the VMs, there's essentially no overhead. If you are in a private cloud, this also creates a good "API boundary" between the team responsible for running hardware, and the team responsible for shipping software to run on that hardware. On the former side you can essentially just adopt a machine into the cluster and leave it, and on the latter side K8s lets you programmatically reference resources, but you don't need to know how/where they came from.
    - jobegrabber 5y ago
      I would say it is an interesting fact, since it's this "good 'API boundary'" that, as you said, enables one to separate concerns, be it between different teams in an organisation or between a service provider and its users. Yes, you don't need Kubernetes to come up with your own implicit or explicit API boundaries, and these might not be needed for smaller projects. I agree that Kubernetes is often used where it's not strictly needed. There are things which strict abstraction, and with it, separation of concerns provide. The crucial point is that certain things are enforced.
      - danpalmer 5y ago
        Sorry, I meant not an interesting fact in public clouds where they take care of all the VMs for you, even without Fargate. You're right though, the abstraction layer is very interesting for _enabling_ public clouds, and for private cloud team ownership.
  - Eritico 5y ago
    Its not the same. You can easily run node pools automatically because your abstraction layer is k8s with containerd or docker. You also know that you can throw away VMs because they don't contain any state. You are not losing data just because you kill a VM or a VM breaks. It is way easier to just spin up n nodes and provision them all equally than whatever you did before. In my team, we can manage way way way more nodes than we ever could. We spin up 100 VMs and destroy them on a regular basis automatically. Gardener for example supports autoscaling on bare metal. The whole ecosystem is providing tons of great options.
    - dijonman2 5y ago
      I think there is a healthy dose of anti-vm bias in your viewpoints. Lauding Kubernetes because ops work took too much of your time is just shifting your burden elsewhere, even if that means paying a bit more for an offering like ECS fargate. Any environment with configuration management can treat instances as ephemeral. It’s a best practice. I view docker more as a package manager, no more dependency hell. In any event K8s is sprawling, it will soon be too complex for its own good. Assuming it’s not already.
      - Eritico 5y ago
        I only answered the question why it is different with kubernetes. I dont have anything against VMs. Feel free to click yourself a VM on any cloud provider, use it however you like. K8s abstracts VMs away and i have and had real issues with maintaining VMs. Docker filling up the node with logs. Unable to upgrade the BaseOS due to python dependencies. Managing the same VM stack through ansible and everything ansible or chef brings to the table. There have been plenty of self healing mechanism in place which do solve unfortunate issues. Memory? The service restarted, was offline for 3 minutes and is now working again. Node disk full? Pods get scheduled away, new node comes up, done. Update/upgrade of nodes? Nodepool does it for me. For me, k8s has 2 real issues like memory (swap support is wip finally!) and stateful workloads like a database. But the concept of an operator shows a bright future. k8s also does one thing very nice: It enforces certain aspects which are a pain in the ass later. That VM which wasn't updated for years and run just fine? Now there is an issue and it needs to be fixed asap. But now the debian repositories are no longer available. I have to fix apt srce list first, then i need to fix dependencies and then i need to restart it.
- gizzlon 5y ago
  That's a PAAS and they existed long before k8
  - danpalmer 5y ago
    A good mental model is to think of K8s as a portable PaaS with a more well-defined API. That's a good thing, not a criticism of K8s.
parkingrift 5y ago
You don’t really have no concern for the underlying hardware. You pick and choose the CPU/GPU horsepower, memory, storage type and storage class, network transfer speed, and many other things. It’s at the point of needing to scale horizontally where I begin to disagree with your premise. This is where you’ll typically get into proprietary and/or ugly offerings.
- dijit 5y ago
  >You pick and choose the CPU/GPU horsepower, memory, storage type and storage class, You still do with kubernetes > network transfer speed, and many other things. No, you don't. There's no slider for "network performance" on GCP, Azure or AWS.
  - nigma1337 5y ago
    There might not be on GCP, but there are on other providers (alibaba cloud comes to mind)
  - lima 5y ago
    Network performance is tied to the instance type on GCP, AWS and presumably Azure.
    - dijit 5y ago
      But there’s no slider. Typically you slide the instance size for other reasons. Here’s the thing: either you care about it (and you can game the sizing of the instance) or you dont and you run kubernetes. But you if you don’t care then it doesn’t matter. There’s no slider for you to care about. It is not extra overhead.
      - nostrebored 5y ago
        why is this slider important? You’ve picked something very arbitrary here
zozbot234 5y ago
The principled way of "abstracting away the server" is in fact namespacing of all OS-managed resources ala containers. This opens up possibilities like automated process checkpointing and migration, or even seamless vertical scaling to a multi-node cluster (as opposed to a single server node) via a SSI (single system image) environment.
- paulryanrogers 5y ago
  Wouldn't your example be horizontal scaling?
- nostrebored 5y ago
  Horizontal scaling of systems is not a way that kubernetes or containerization benefits over the cloud. Resource utilization on those horizontally scaled workers can be, however.
danpalmer 5y ago
There are several layers of abstraction here, all bringing benefits. Layer 1: bare metal servers, bare metal routers, hardware appliances, etc. Layer 2: virtual servers, VLANs/security groups/etc, some appliances. Layer 3: containers, container networks, storage resources, etc. Kubernetes, arguably Heroku. Layer 4: functions as a service? In the article the suggestion, or how I read it, is essentially "Kubernetes is nothing new, we've had [layer 2] for years". However I see Kubernetes as sitting squarely in "layer 3" here. These layers are pretty subjective, all of the boundaries are blurred, but when I'm working these do require fairly different skills sets. I've never had the skillset to work at "1", I was fairly good at "2" for a while, but spent too much of my time working on that and too little shipping software, so my workplace moved to Kubernetes and it allowed me as an application engineer to do much less infra work, and for it to be at "3" when necessary. To just write off everything in "3" as being "virtualisation" is ignoring the significant step in level of abstractions and the benefits brought by that.
- zozbot234 5y ago
  Process boundaries do imply a kind of virtualization, OP is not wrong there. What containers add as a feature though is comprehensive namespacing for the resources that the OS manages on behalf of "virtualized" processes.
  - nostrebored 5y ago
    But the initial question was how it differs from public cloud. This is not a difference. You can define your kubernetes or your terraform and have whichever brand of logical isolation you prefer
Eritico 5y ago
I can tell you what it does for me: Full disk on your VM? Nope. Storage is abstracted away. Your server will not fill up anymore. Only one service might break which might heal itself. Is your VM/node broken? It will heal itself because you throw it away and the new VM/node is fixed. It enforces the abstraction of Service and VM. You will not install normal software on that VM just because you can. You don't need to give access to a VM to a developer who then needs root access and has dependencies and doesn't update the VM. You no longer have dependencies to your VM because you can't have dependencies on your VM OS. Abstracting it away from your VM also streamlines things like logfiles. You no longer need to collect all logfiles from VMs because you do it for your services and for your services, you only do it once (if even, log to stdout and be done with it)
- withinboredom 5y ago
  None of this is true.
  - Eritico 5y ago
    I'm describing my real life issues i have and had. Feel free to actually write more than 'None of this is true.' in a way that a discussion is actually possible. Tx :)
    - withinboredom 5y ago
      I run k8s on bare-metal, and I can say a full disk is certainly possible if you have a service logging a few mb/s. Things will break in fun and interesting ways, data will get irrecoverably corrupted, etc. Your entire cluster will probably even break if said node was the etcd leader. This is pretty easy to reproduce by simply saturating a network and then watching the etcd leader spill its guts in your logs once the network buffers fill up. > You no longer have dependencies to your VM because you can't have dependencies on your VM OS. Your containers rely on the OS's kernel and whatever features it was compiled with. > You will not install normal software on that VM just because you can If you're paying through the nose for managed k8s, this is true. If not, you'll eventually need to login to a node and diagnose some issue, which means installing things on the node. > You no longer need to collect all logfiles from VMs because you do it for your services and for your services Whatever you installed to collect logfiles is getting them from the VM's disk (in /var/log/pods in k3s), unless your container is redirecting them somewhere that isn't stdout.
throwaway894345 5y ago
With Kubernetes you don't have to configure log exfiltration, process management, SSH, host metrics, etc. You don't have to touch Ansible--there's no host management at all. The stuff that you still have to configure (e.g., firewalls, NFS) is all configured through a consistent, declarative interface (Kubernetes manifests) rather than a dozen bespoke, byzantine formats or imperative commands.
- chousuke 5y ago
  Kubernetes is not quite that easy though. Out of the box, you get basically no isolation between anything, and you still have to deal with security contexts and have processes in place for keeping your container images secure. If you use community Helm charts your services may end up running with essentially random privileges that may easily conflict. The declarative interface is going on the right direction (as far as yaml can be) but configuration management for it is still unsolved. Backups are also often forgotten; they're very easy with virtual machines.
  - throwaway894345 5y ago
    I suspect you may be confusing "cloud provider Kubernetes" (the topic at hand) with running your own Kubernetes on bare metal. The bare metal Kubernetes story still has a long way to go, but we're talking about public cloud providers. > Out of the box, you get basically no isolation between anything I'm pretty sure AWS Fargate and GCP's GVisor solve (or attempt to solve) isolation. Not sure about other cloud providers. > you still have to deal with security contexts and have processes in place for keeping your container images secure How do VMs help secure software artifacts beyond the security practices in the container ecosystem? And I would argue that "dealing with security contexts" is strictly better in Kubernetes than the equivalent in VMs if only because of the unified interface (Kubernetes manifests). > If you use community Helm charts your services may end up running with essentially random privileges that may easily conflict. You can run into the same issue with Ansible scripts on VMs. This isn't a Kubernetes specific issue--ultimately, all system administrators need to take care to run secure software on their systems. Neither Kubernetes nor VMs offer a silver bullet here. > configuration management for it is still unsolved If "configuration management" refers to configuration of the hosts, then yes, public cloud provider Kubernetes offerings solve for this--you don't have to manage the host configuration at all (unless you want to opt into it). > Backups are also often forgotten; they're very easy with virtual machines. The etcd backups are managed by the cloud providers, as are backups for mounted volumes. Not sure what backups you're thinking about.
koffiezet 5y ago
It's not just "abstracting the server". Kubernetes abstracts more than just "a server", it works on a level higher. It does this for storage, networking, compute, services, workloads, scaling, ... and all this is done through a standardised API. This APIforces you to standardise application deployments, making centrally managed logging, monitoring, tracing, ... a breeze. Once you have it working for one application, it'll work for all of them. And do you need to run this on Amazon, Gcloud, Azure,one of the smaller cloud providers like DigitalOcean or locally on Kind/k3s? It'll require very little work to get them working on any of these - if any. Cloud specific services and persistent storage will be the main issues, but that's something you can't really get around. Now is it perfect? Absolutely not, as with any tech, there will always be problems and bottlenecks. But it allows development to scale, not just the workloads, and the skills required are transferable, which makes it a much easier sell.
- freedomben 5y ago
  I completely agree with everything you said, except: > Cloud specific services and persistent storage will be the main issues, but that's something you can't really get around. That isn't wrong necessarily, but products like OpenShift Container Storage (OpenShift Data Foundations now actually) can provide a common API to erase that problem. ODF uses Ceph under the hood so you can get block, file, and (s3 compatible) object storage no matter where you are. Cloud specific services are indeed a problem, but many of them have open source/portable solutions that you an choose that can run everywhere. Such as Fission, RabbitMQ, Kafka (not my favorite), Argo CD, etc. Really the things I run into most now are things like AWS machine learning services.

charcircuit 5y ago

I think part of the confusion is that server can refer to both a physical box and a VM. Abstracting away servers is about not having to think about VMs. You have a service which requires some amount of compute, memory, and storage and you just want that service to run. There is value to not having to worry about provisioning a VM or administering it.

jasonrichardsmi 5y ago
But you still have to do that in Kubernetes, unless you are running Fargate. Someone has to provision and maintain that machine, and in the process introduce a ton of administrative overhead.
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- danpalmer 5y ago
  That's true, but not a particularly interesting fact. For example, if you use GKE (Google Cloud's K8s offering), you attach your K8s cluster to an auto-scaling node pool and it handles (de)provisioning of your VMs for you. You essentially don't care about the VMs, there's essentially no overhead. If you are in a private cloud, this also creates a good "API boundary" between the team responsible for running hardware, and the team responsible for shipping software to run on that hardware. On the former side you can essentially just adopt a machine into the cluster and leave it, and on the latter side K8s lets you programmatically reference resources, but you don't need to know how/where they came from.
  - jobegrabber 5y ago
    I would say it is an interesting fact, since it's this "good 'API boundary'" that, as you said, enables one to separate concerns, be it between different teams in an organisation or between a service provider and its users. Yes, you don't need Kubernetes to come up with your own implicit or explicit API boundaries, and these might not be needed for smaller projects. I agree that Kubernetes is often used where it's not strictly needed. There are things which strict abstraction, and with it, separation of concerns provide. The crucial point is that certain things are enforced.
    - danpalmer 5y ago
      Sorry, I meant not an interesting fact in public clouds where they take care of all the VMs for you, even without Fargate. You're right though, the abstraction layer is very interesting for _enabling_ public clouds, and for private cloud team ownership.
- Eritico 5y ago
  Its not the same. You can easily run node pools automatically because your abstraction layer is k8s with containerd or docker. You also know that you can throw away VMs because they don't contain any state. You are not losing data just because you kill a VM or a VM breaks. It is way easier to just spin up n nodes and provision them all equally than whatever you did before. In my team, we can manage way way way more nodes than we ever could. We spin up 100 VMs and destroy them on a regular basis automatically. Gardener for example supports autoscaling on bare metal. The whole ecosystem is providing tons of great options.
  - dijonman2 5y ago
    I think there is a healthy dose of anti-vm bias in your viewpoints. Lauding Kubernetes because ops work took too much of your time is just shifting your burden elsewhere, even if that means paying a bit more for an offering like ECS fargate. Any environment with configuration management can treat instances as ephemeral. It’s a best practice. I view docker more as a package manager, no more dependency hell. In any event K8s is sprawling, it will soon be too complex for its own good. Assuming it’s not already.
    - Eritico 5y ago
      I only answered the question why it is different with kubernetes. I dont have anything against VMs. Feel free to click yourself a VM on any cloud provider, use it however you like. K8s abstracts VMs away and i have and had real issues with maintaining VMs. Docker filling up the node with logs. Unable to upgrade the BaseOS due to python dependencies. Managing the same VM stack through ansible and everything ansible or chef brings to the table. There have been plenty of self healing mechanism in place which do solve unfortunate issues. Memory? The service restarted, was offline for 3 minutes and is now working again. Node disk full? Pods get scheduled away, new node comes up, done. Update/upgrade of nodes? Nodepool does it for me. For me, k8s has 2 real issues like memory (swap support is wip finally!) and stateful workloads like a database. But the concept of an operator shows a bright future. k8s also does one thing very nice: It enforces certain aspects which are a pain in the ass later. That VM which wasn't updated for years and run just fine? Now there is an issue and it needs to be fixed asap. But now the debian repositories are no longer available. I have to fix apt srce list first, then i need to fix dependencies and then i need to restart it.
gizzlon 5y ago
That's a PAAS and they existed long before k8
- danpalmer 5y ago
  A good mental model is to think of K8s as a portable PaaS with a more well-defined API. That's a good thing, not a criticism of K8s.

parkingrift 5y ago

You don’t really have no concern for the underlying hardware. You pick and choose the CPU/GPU horsepower, memory, storage type and storage class, network transfer speed, and many other things. It’s at the point of needing to scale horizontally where I begin to disagree with your premise. This is where you’ll typically get into proprietary and/or ugly offerings.

dijit 5y ago
>You pick and choose the CPU/GPU horsepower, memory, storage type and storage class, You still do with kubernetes > network transfer speed, and many other things. No, you don't. There's no slider for "network performance" on GCP, Azure or AWS.
- nigma1337 5y ago
  There might not be on GCP, but there are on other providers (alibaba cloud comes to mind)
- lima 5y ago
  Network performance is tied to the instance type on GCP, AWS and presumably Azure.
  - dijit 5y ago
    But there’s no slider. Typically you slide the instance size for other reasons. Here’s the thing: either you care about it (and you can game the sizing of the instance) or you dont and you run kubernetes. But you if you don’t care then it doesn’t matter. There’s no slider for you to care about. It is not extra overhead.
    - nostrebored 5y ago
      why is this slider important? You’ve picked something very arbitrary here

zozbot234 5y ago

The principled way of "abstracting away the server" is in fact namespacing of all OS-managed resources ala containers. This opens up possibilities like automated process checkpointing and migration, or even seamless vertical scaling to a multi-node cluster (as opposed to a single server node) via a SSI (single system image) environment.

paulryanrogers 5y ago
Wouldn't your example be horizontal scaling?
nostrebored 5y ago
Horizontal scaling of systems is not a way that kubernetes or containerization benefits over the cloud. Resource utilization on those horizontally scaled workers can be, however.

danpalmer 5y ago

There are several layers of abstraction here, all bringing benefits. Layer 1: bare metal servers, bare metal routers, hardware appliances, etc. Layer 2: virtual servers, VLANs/security groups/etc, some appliances. Layer 3: containers, container networks, storage resources, etc. Kubernetes, arguably Heroku. Layer 4: functions as a service? In the article the suggestion, or how I read it, is essentially "Kubernetes is nothing new, we've had [layer 2] for years". However I see Kubernetes as sitting squarely in "layer 3" here. These layers are pretty subjective, all of the boundaries are blurred, but when I'm working these do require fairly different skills sets. I've never had the skillset to work at "1", I was fairly good at "2" for a while, but spent too much of my time working on that and too little shipping software, so my workplace moved to Kubernetes and it allowed me as an application engineer to do much less infra work, and for it to be at "3" when necessary. To just write off everything in "3" as being "virtualisation" is ignoring the significant step in level of abstractions and the benefits brought by that.

zozbot234 5y ago
Process boundaries do imply a kind of virtualization, OP is not wrong there. What containers add as a feature though is comprehensive namespacing for the resources that the OS manages on behalf of "virtualized" processes.
- nostrebored 5y ago
  But the initial question was how it differs from public cloud. This is not a difference. You can define your kubernetes or your terraform and have whichever brand of logical isolation you prefer

Eritico 5y ago

I can tell you what it does for me: Full disk on your VM? Nope. Storage is abstracted away. Your server will not fill up anymore. Only one service might break which might heal itself. Is your VM/node broken? It will heal itself because you throw it away and the new VM/node is fixed. It enforces the abstraction of Service and VM. You will not install normal software on that VM just because you can. You don't need to give access to a VM to a developer who then needs root access and has dependencies and doesn't update the VM. You no longer have dependencies to your VM because you can't have dependencies on your VM OS. Abstracting it away from your VM also streamlines things like logfiles. You no longer need to collect all logfiles from VMs because you do it for your services and for your services, you only do it once (if even, log to stdout and be done with it)

withinboredom 5y ago
None of this is true.
- Eritico 5y ago
  I'm describing my real life issues i have and had. Feel free to actually write more than 'None of this is true.' in a way that a discussion is actually possible. Tx :)
  - withinboredom 5y ago
    I run k8s on bare-metal, and I can say a full disk is certainly possible if you have a service logging a few mb/s. Things will break in fun and interesting ways, data will get irrecoverably corrupted, etc. Your entire cluster will probably even break if said node was the etcd leader. This is pretty easy to reproduce by simply saturating a network and then watching the etcd leader spill its guts in your logs once the network buffers fill up. > You no longer have dependencies to your VM because you can't have dependencies on your VM OS. Your containers rely on the OS's kernel and whatever features it was compiled with. > You will not install normal software on that VM just because you can If you're paying through the nose for managed k8s, this is true. If not, you'll eventually need to login to a node and diagnose some issue, which means installing things on the node. > You no longer need to collect all logfiles from VMs because you do it for your services and for your services Whatever you installed to collect logfiles is getting them from the VM's disk (in /var/log/pods in k3s), unless your container is redirecting them somewhere that isn't stdout.

throwaway894345 5y ago

With Kubernetes you don't have to configure log exfiltration, process management, SSH, host metrics, etc. You don't have to touch Ansible--there's no host management at all. The stuff that you still have to configure (e.g., firewalls, NFS) is all configured through a consistent, declarative interface (Kubernetes manifests) rather than a dozen bespoke, byzantine formats or imperative commands.

chousuke 5y ago
Kubernetes is not quite that easy though. Out of the box, you get basically no isolation between anything, and you still have to deal with security contexts and have processes in place for keeping your container images secure. If you use community Helm charts your services may end up running with essentially random privileges that may easily conflict. The declarative interface is going on the right direction (as far as yaml can be) but configuration management for it is still unsolved. Backups are also often forgotten; they're very easy with virtual machines.
- throwaway894345 5y ago
  I suspect you may be confusing "cloud provider Kubernetes" (the topic at hand) with running your own Kubernetes on bare metal. The bare metal Kubernetes story still has a long way to go, but we're talking about public cloud providers. > Out of the box, you get basically no isolation between anything I'm pretty sure AWS Fargate and GCP's GVisor solve (or attempt to solve) isolation. Not sure about other cloud providers. > you still have to deal with security contexts and have processes in place for keeping your container images secure How do VMs help secure software artifacts beyond the security practices in the container ecosystem? And I would argue that "dealing with security contexts" is strictly better in Kubernetes than the equivalent in VMs if only because of the unified interface (Kubernetes manifests). > If you use community Helm charts your services may end up running with essentially random privileges that may easily conflict. You can run into the same issue with Ansible scripts on VMs. This isn't a Kubernetes specific issue--ultimately, all system administrators need to take care to run secure software on their systems. Neither Kubernetes nor VMs offer a silver bullet here. > configuration management for it is still unsolved If "configuration management" refers to configuration of the hosts, then yes, public cloud provider Kubernetes offerings solve for this--you don't have to manage the host configuration at all (unless you want to opt into it). > Backups are also often forgotten; they're very easy with virtual machines. The etcd backups are managed by the cloud providers, as are backups for mounted volumes. Not sure what backups you're thinking about.

koffiezet 5y ago

It's not just "abstracting the server". Kubernetes abstracts more than just "a server", it works on a level higher. It does this for storage, networking, compute, services, workloads, scaling, ... and all this is done through a standardised API. This APIforces you to standardise application deployments, making centrally managed logging, monitoring, tracing, ... a breeze. Once you have it working for one application, it'll work for all of them. And do you need to run this on Amazon, Gcloud, Azure,one of the smaller cloud providers like DigitalOcean or locally on Kind/k3s? It'll require very little work to get them working on any of these - if any. Cloud specific services and persistent storage will be the main issues, but that's something you can't really get around. Now is it perfect? Absolutely not, as with any tech, there will always be problems and bottlenecks. But it allows development to scale, not just the workloads, and the skills required are transferable, which makes it a much easier sell.

freedomben 5y ago
I completely agree with everything you said, except: > Cloud specific services and persistent storage will be the main issues, but that's something you can't really get around. That isn't wrong necessarily, but products like OpenShift Container Storage (OpenShift Data Foundations now actually) can provide a common API to erase that problem. ODF uses Ceph under the hood so you can get block, file, and (s3 compatible) object storage no matter where you are. Cloud specific services are indeed a problem, but many of them have open source/portable solutions that you an choose that can run everywhere. Such as Fission, RabbitMQ, Kafka (not my favorite), Argo CD, etc. Really the things I run into most now are things like AWS machine learning services.