5 ms·

One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so. Also, when r

by lyptt 6y ago

One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so. Also, when rolling your own cluster it requires a lot of elbow grease to get things in a working state, and even more to keep things working. It feels awesome from a developer standpoint once you work out all the kinks. It's very natural describing your service with a bit of yaml, and then being able to then spawn it into your infrastructure and have it automatically scale out. If there was a cost effective k8s cluster you could rent then that would be the thing that finally kills traditional infrastructure for me.

rhacker 6y ago
That's the exact same feeling I get too. If I'm a mom/pop shop online, why can't I get a k8s cluster w/just 1 machine for $5/month? I mean the big three (4 w DO) can all get you a bare machine for $5/mo that can run Docker. Why not give the k8s management side for free for those people?
- throwaway201103 6y ago
  If you're mom/pop online shop and just need 1 machine you don't need k8s. KISS principle applies.
  - rhacker 6y ago
    Well, in this case the pop is me, and I know k8s but I don't want to invest 15-35 / month on a side project but would still rather have a decently managed site rather. I don't want to go back to FTP crap at this point. A car is a hugely complicated machine and I'm sure in the 1910s people said only some people will ever need them. Bad analogy considering cars kill lots of people but take that part out :)
  - bpodgursky 6y ago
    A ton of the value from k8s is in the programmatic and replicable configuration. 1 computer pets are still bad in a mom and pop shop. 1 computer livestock is still better.
    - throwaway894345 6y ago
      Not to mention everything you have to do to get a machine to parity with Kubernetes--set up SSH, process management, monitoring, logging, etc. I don't think I'd go so far as to say Kubernetes is the right tool for this use case, but I certainly empathize with the sentiment.
    - kitotik 6y ago
      Is “1 computer livestock” even possible? After your single cattle is murdered, the next one has to be raised right? State has to come from somewhere. Downtime has a cost for most businesses.
      - bpodgursky 6y ago
        Downtime has cost. But being able to blow away a k8s node and bring it back up in 2 minutes (or have k8s auto-spawn a replacement node) is WAY faster than having to reinstall wordpress, reinstall nginx, set IP addresses, fiddle with your DNS, restore a database from a backup, etc, etc, etc.
      - aaomidi 6y ago
        In this case the state can come from a centrally managed k8s controller. Something that most cloud providers provide and have good SLAs on. Kubernetes just makes that story so much simpler. Heck a good cloud provider can just handle any machines fucking up automatically without any human intervention on your side. The same can't be said for a single machine.
    - twelve40 6y ago
      so... for a small cluster then it's like aws beanstalk, right? rolling deployments, load balancing, etc. all for free.
  - WJW 6y ago
    From my experience, deploying to a k8s cluster managed by someone else is extremely simple and straightforward. Tell it which container to run and off it goes. You get zero-downtime rolling deploys, automatic restarts on crashes, easy rollbacks, service discovery (if you have multiple services) and more for free. Managing a cluster is the tricky part of k8s but just running your stuff on an existing cluster is extremely straightforward. I could definitely see the attraction of a cheap managed k8s cluster with a single node. Maybe hosted k3s or something.
- OJFord 6y ago
  Because that's more the realm of: https://www.digitalocean.com/products/kubernetes/ https://www.digitalocean.com/products/kubernetes/ https://www.scaleway.com/en/kubernetes-kapsule/ https://www.scaleway.com/en/kubernetes-kapsule/ IaaS model is more 'everything is chargeable in small units', a free control plane doesn't really gel with that IMO, and also they have some huge users that it'd cost a lot to run the control plane for, pushing up the price of what you connect to it (or something must give) and making more PaaS-style like DO & Scaleway anyway.
- algeoMA 6y ago
  You can get a small cluster from Digital Ocean pretty cheaply. It's more than $5 (I forget the exact price) but they'll provide a free master node and then you just pay for one additional kubelet (and you can add more as you need them). I have used it for one hobby project and have been pretty happy with it.
  - rhacker 6y ago
    I'll have to check that out!
  - geerlingguy 6y ago
    Same with Linode, the management machines are not massive scale HA control planes like you get with the expensive hosts (It seems like Linode may run a single 2GB droplet), but for most of my clusters I don't need it.
- dcolkitt 6y ago
  You can run a single-core preemptible instance on GKE for $7/month. As long as you keep the one instance alive that persists the cluster.
- comradesmith 6y ago
  You can with digital ocean
- renewiltord 6y ago
  Get the zonal cluster on GKE. Free control plane stuff. Then you can put as many or as few nodes as you want on it. Make sure to use metal LB or all your ingresses will be served with Google LBs that are hecka expensive.
k__ 6y ago
Haha. I forgot to tear down a small EKS cluster last month. 4 days, 60€.
flowerlad 6y ago
> One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so. I run a 3-node Kubernetes cluster on Google cloud. It costs me around $100 per month (with 1-year reserved instances). Not terribly expensive.
- crb002 6y ago
  What is your utilization? IMHO I would rather do one EC2 instance then have EC2 Batch spot and lambda workers take the other load. In a good design most media assets should never hit the full time provisioned server.
area51org 6y ago
And, not everything should be in a container. Just because you can does not necessarily mean you should. Containers are awesome but they are not a cure-all silver bullet.

by lyptt 6y ago

rhacker 6y ago

That's the exact same feeling I get too. If I'm a mom/pop shop online, why can't I get a k8s cluster w/just 1 machine for $5/month? I mean the big three (4 w DO) can all get you a bare machine for $5/mo that can run Docker. Why not give the k8s management side for free for those people?

throwaway201103 6y ago
If you're mom/pop online shop and just need 1 machine you don't need k8s. KISS principle applies.
- rhacker 6y ago
  Well, in this case the pop is me, and I know k8s but I don't want to invest 15-35 / month on a side project but would still rather have a decently managed site rather. I don't want to go back to FTP crap at this point. A car is a hugely complicated machine and I'm sure in the 1910s people said only some people will ever need them. Bad analogy considering cars kill lots of people but take that part out :)
- bpodgursky 6y ago
  A ton of the value from k8s is in the programmatic and replicable configuration. 1 computer pets are still bad in a mom and pop shop. 1 computer livestock is still better.
  - throwaway894345 6y ago
    Not to mention everything you have to do to get a machine to parity with Kubernetes--set up SSH, process management, monitoring, logging, etc. I don't think I'd go so far as to say Kubernetes is the right tool for this use case, but I certainly empathize with the sentiment.
  - kitotik 6y ago
    Is “1 computer livestock” even possible? After your single cattle is murdered, the next one has to be raised right? State has to come from somewhere. Downtime has a cost for most businesses.
    - bpodgursky 6y ago
      Downtime has cost. But being able to blow away a k8s node and bring it back up in 2 minutes (or have k8s auto-spawn a replacement node) is WAY faster than having to reinstall wordpress, reinstall nginx, set IP addresses, fiddle with your DNS, restore a database from a backup, etc, etc, etc.
    - aaomidi 6y ago
      In this case the state can come from a centrally managed k8s controller. Something that most cloud providers provide and have good SLAs on. Kubernetes just makes that story so much simpler. Heck a good cloud provider can just handle any machines fucking up automatically without any human intervention on your side. The same can't be said for a single machine.
  - twelve40 6y ago
    so... for a small cluster then it's like aws beanstalk, right? rolling deployments, load balancing, etc. all for free.
- WJW 6y ago
  From my experience, deploying to a k8s cluster managed by someone else is extremely simple and straightforward. Tell it which container to run and off it goes. You get zero-downtime rolling deploys, automatic restarts on crashes, easy rollbacks, service discovery (if you have multiple services) and more for free. Managing a cluster is the tricky part of k8s but just running your stuff on an existing cluster is extremely straightforward. I could definitely see the attraction of a cheap managed k8s cluster with a single node. Maybe hosted k3s or something.
OJFord 6y ago
Because that's more the realm of: https://www.digitalocean.com/products/kubernetes/ https://www.digitalocean.com/products/kubernetes/ https://www.scaleway.com/en/kubernetes-kapsule/ https://www.scaleway.com/en/kubernetes-kapsule/ IaaS model is more 'everything is chargeable in small units', a free control plane doesn't really gel with that IMO, and also they have some huge users that it'd cost a lot to run the control plane for, pushing up the price of what you connect to it (or something must give) and making more PaaS-style like DO & Scaleway anyway.
algeoMA 6y ago
You can get a small cluster from Digital Ocean pretty cheaply. It's more than $5 (I forget the exact price) but they'll provide a free master node and then you just pay for one additional kubelet (and you can add more as you need them). I have used it for one hobby project and have been pretty happy with it.
- rhacker 6y ago
  I'll have to check that out!
- geerlingguy 6y ago
  Same with Linode, the management machines are not massive scale HA control planes like you get with the expensive hosts (It seems like Linode may run a single 2GB droplet), but for most of my clusters I don't need it.
dcolkitt 6y ago
You can run a single-core preemptible instance on GKE for $7/month. As long as you keep the one instance alive that persists the cluster.
comradesmith 6y ago
You can with digital ocean
renewiltord 6y ago
Get the zonal cluster on GKE. Free control plane stuff. Then you can put as many or as few nodes as you want on it. Make sure to use metal LB or all your ingresses will be served with Google LBs that are hecka expensive.

k__ 6y ago

Haha. I forgot to tear down a small EKS cluster last month. 4 days, 60€.

flowerlad 6y ago

> One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so. I run a 3-node Kubernetes cluster on Google cloud. It costs me around $100 per month (with 1-year reserved instances). Not terribly expensive.

crb002 6y ago
What is your utilization? IMHO I would rather do one EC2 instance then have EC2 Batch spot and lambda workers take the other load. In a good design most media assets should never hit the full time provisioned server.

area51org 6y ago

And, not everything should be in a container. Just because you can does not necessarily mean you should. Containers are awesome but they are not a cure-all silver bullet.