12 ms·

Managed Kubernetes Price Comparison

168 points by spalas 7y ago

oroup 7y ago
At the low end it’s worth considering Fargate distinct from EKS. You don’t need to provision a whole cluster (generally 3 machines minimum) and can just run as little as a single Pod.
- petilon 7y ago
  I tried Fargate and found it to be crappy. It is very hard to use. It is proprietary, so your app will not be portable, and your knowledge and experience will not be portable either. If you use Kubernetes there is tons of tutorials, your app becomes portable across clouds and your knowledge is portable from cloud to cloud too. GKE only costs around $60 per month for a single-machine "cluster".
  - txcwpalpha 7y ago
    What is proprietary about Fargate? It's containers. I did not find any experience/knowledge (other than the basic knowledge of navigating the AWS console) that wouldn't transfer to any other container service.
    - petilon 7y ago
      AWS console is the crappy part. Azure and Google have much better GUIs. And here's the proprietary part: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/... For contrast, you can manage a Kubernetes deployment using standardized yaml and kubectl commands, regardless of whether the application is running on localhost (minikube), on Azure or on GKE. BTW, AWS Lightsail has decent GUI. Alas, it doesn't support containers out of the box. The best support for Docker image-based deployment is Azure App Service.
      - bdcravens 7y ago
        > here's the proprietary part: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/.. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/.... That's ECS, not EKS. Two different products. The EKS documentation is at https://docs.aws.amazon.com/eks/latest/userguide/fargate.html https://docs.aws.amazon.com/eks/latest/userguide/fargate.htm... > For contrast, you can manage a Kubernetes deployment using standardized yaml and kubectl commands, regardless of whether the application is running on localhost (minikube), on Azure or on GKE. Likewise for EKS.
        petilon 7y ago
        I was replying to this: > At the low end it’s worth considering Fargate distinct from EKS.
        txcwpalpha 7y ago
        Right, and you linked to the documentation for ECS on Fargate rather than the documentation for Kubernetes Fargate, which is what was being talked about. Again, two different products.
      - txcwpalpha 7y ago
        I'm still not seeing the difference. As pointed out, what you linked is for ECS. That has nothing to do with Kubernetes, so I'm not sure why you're comparing the things on that page to kubectl commands on GKE or Azure. Of course you cannot use kubectl on ECS, because ECS has nothing to do with kube. When you are using actual EKS (with or without Fargate), you certainly can use standardized kubectl commands. The only "proprietary" things I see in your link is the specific AWS CLI commands used to set up the cluster before you can use kubectl, but both Azure and GCP require using the Azure CLI and gcloud CLI for cluster deployment, too. There's also setting up AWS-specific security groups and IAM roles, but you have to do those same things on GCP or Azure, too, and both of those have their own "proprietary" ways of setting up networking and security, so I don't see the differentiating factor.
  - thoraway1010 7y ago
    I use fargate and pretty happy with it. Don't need big scale out - it supports $1M/year revenue so not huge, but LOVE the simplicity. I just have the CLI commands in my dockerfiles as comments, so once I get things sorted locally using docker I update the task with some copy / paste. I only update occasionally when I need to make some changes (locally do a lot more). The one thing I'd love to get my DOCKER image sizes down - they seem way too big for what they do but it's just easier to start with full fat images. I tried alpine images and couldn't get stuff to install / compile etc.
    - grey 7y ago
      You should look into multistage docker builds, that lets you still use a full fat image for your build but then leave all the build tools out of your final image I liked jpetazzo's post on the subject but there are plenty to choose from https://www.ardanlabs.com/blog/2020/02/docker-images-part1-reducing-image-size.html https://www.ardanlabs.com/blog/2020/02/docker-images-part1-r...
      - thoraway1010 7y ago
        Someone else suggested the same thing actually. Easy to get lazy when it "just works" and internet is 1gig home and office - you can see how bloat just builds up.
  - scarface74 7y ago
    How is Fargate “proprietary”? It runs a Docker containers. But if you are at any type of scale, the last thing on your to do list is “cloud migration”.
0x1221 7y ago
I'm not familiar with the space so my question might not be that relevant - where does OpenShift fit in all of this (I still struggle to differentiate it from Kubernetes) and is there any merit to IBM trying to sell it so hard?
- symfrog 7y ago
  OpenShift is Kubernetes, just like RHEL is a Linux distribution with support for enterprises. OpenShift makes an opinionated choice about what they bundle (distribute) with vanilla Kubernetes. For example, Istio was chosen as the service mesh distributed with OpenShift 4.
  - freedomben 7y ago
    I agree with you here, but do think it's worth pointing out that OpenShift adds some non-trivial features to Kubernetes and they aren't just plucked from existing projects and bundled. Some major K8s features started out as OpenShift features and got merged upstream (Ingress, Deployments, etc). Red Hat innovates and improves the distribution rather than just repackaging it and slapping on a support contract.
- p_l 7y ago
  Openshift wraps around Kubernetes, with some of their own special offering stuff on it. Generally, plain K8s is a building block - RedHat made Openshift with a bunch of opinionated choices, geared towards enterprise deployments, some of them migrating later to Kubernetes itself (OpenShift's Route inspired K8s' Ingress), some OpenShift cribs from K8s (Istio becoming part of OpenShift by default in OS 4). Generally OpenShift heavily targets enterprises as "All-in-One" package. Some of that works, some doesn't, but honestly it's often more a case of the IT dept that manages the install ;) Except installing OpenShift. That's horrific. Someone should repent for the install process, seriously.
  - Conan_Kudo 7y ago
    Even with OpenShift 4? I thought it was pretty nice and straightforward, to be honest...
david-s 7y ago
It doesn't seem to include Digital Ocean in the comparison.
- axaxs 7y ago
  DO is my absolute favorite. I really think they could be a long term winner. Their interface is so much nicer than the competitors, in my opinion. I'm not even currently a customer, let alone a shill.
  - steve_adams_86 7y ago
    I agree that DO is awesome. I'd argue though that they can make a better UI because they offer less. Everything is a littler simpler. It would be hard to condense AWS into a similar type of interface. Having said that, DO is enough for virtually everything I've ever worked on, and the user experience and price are so much better. They're a clear winner for almost everything I do these days.
    - axaxs 7y ago
      I agree but that's part of the charm to me. I only use what 4 or 5 things in AWS, but each login is information overload. Having to Ctrl F what you are looking for is not an ideal experience. Whether a conscious decision or not, I think offering what the 80ish percent (just a guess) actually use, and streamlining it, is the right decision.
    - freehunter 7y ago
      AWS could really use a dashboard where you can pick the components you want to see and only those show up. If I only use S3 and EC2 I shouldn’t have to search for those two products every time I log in.
      - scarface74 7y ago
        The console already shows the five most recently used components when you log in.
almostdigital 7y ago
Scaleway gives you a k8s cluster starting from 40 EUR per month
- xbenjii 7y ago
  They allow the DEV1-M boxes now, which are 8EUR a month.
- neop1x 7y ago
  Thanks for this! I've looked at Scaleway and so far I like it a lot, this looks like the end of my months of suffering. It seems to fit my needs perfectly. Finally a smaller cloud provider doing it right!
mcdoker18 7y ago
Only Azure doesn’t charge for the k8s control plane, that is the most surprising thing for me.
- petilon 7y ago
  We don't know how long that's going to last. Google didn't charge for control plane either, until recently. So Azure couldn't charge, in order to be competitive. Now that Google has started charging Azure may start too.
  - fernandotakai 7y ago
    Azure also doesn't have an SLA, so that's why they don't charge. Google started charging after they added an SLA.
    - dodobirdlord 7y ago
      Makes sense, the point of an SLA is that you agree to pay back the customer's money if you don't meet it. What does it mean to have an SLA for a free product?
spectramax 7y ago
Just curious - what's wrong with buying a large instance (24 cores) and running it for < 10,000 users? Kubernetes feels like an insane complexity that doesn't need to be taken on and managed. You're gonna spend more time managing Kubernetes than writing actual software. Also, it feels like if something goes wrong in prod with your cluster - you're gonna need external help to get you back on the feet. If you're not going to build the next Facebook, why would you need so much complexity?
- halfmatthalfcat 7y ago
  Because it removes both provisioning and deployment concerns when you can build a container and then just tell Kubernetes to deploy it onto the cluster. Theres not much that goes into spinning up these managed Kuberentes clusters. Most of it is telling them what classes of instances you want created. When you buy a large instance, you still need to set up the instance and tweak it to your application's needs. You then need to babysit this node.
  - warrenm 7y ago
    >When you buy a large instance, you still need to set up the instance and tweak it to your application's needs. You then need to babysit this node. It's called "use kickstart"
- eknkc 7y ago
  It is a single point of failure. I want multiple smaller instances even if I’m running something trivial. It is not about scale but about reliability for us. And the moment I go with multiple instances I need something to manage that mess. Kube handles it well.
  - spectramax 7y ago
    I've used DigitalOcean's load balancer + multiple instances that run app containers. It works fine and without Kubernetes. I wasn't saying that just use 24 core single instance. Perhaps I should have worded it better.
madjam002 7y ago
Too bad AKS is just terrible. Slow provisioning time, slow PVCs, slow LoadBalancer provisioning, slow node pool management, plus non-production ready node pool implementation.
- aliswe 7y ago
  Agreed, not below usable though. Some more: rolling upgrades of k8s (said to not affect the uptime of the cluster) not being rolling in actuality, allowing upgrades when the service principal is expired thus preventing the nodes from being added to the LB, certain aks versions not being upgradable requiring you to recreate the cluster from scratch ...
neop1x 7y ago
Egress costs at those major clouds are ridiculous. Once you start having some traffic it can easily make 50% of all costs or more. That is not justifiable! Meanwhile hardware costs are going down. We need more k8s providers with more resonable pricing. Unfortunatelly both Digital Ocean and Oracle Cloud don't have proper network load balancer implementations which is a must for elastic regional clusters and to forward TCP in a way that client IP is preserved and be able to add nodes without downtimes or TCP resets. OVH cloud doesn't implement LoadBalancer service type at all. So the choice in 2020 is really just Google, Amazon, Azure with their rolls-royce pricing. The cost difference between them is neglible. And then there are confidential free credits for startups. So sad...
- atombender 7y ago
  For HTTP, an external load balancer/CDN such as CloudFlare or Fastly could easily fulfill that role, though you obviously have to jump through a few more hoops to get there, since it's not built in. But with some of them you also get things like TLS termination, DNS hosting, advanced rewriting, caching, and DDOS protection.
- ikiris 7y ago
  You seem to think theres some kind of disconnect between egress pricing, and if the network has a functional backbone with load balancing infrastrucutre. If anything, egress pricing is to low in the vast majority of cases, and it really shows when customers start using it.
- vasco 7y ago
  All of AWS's VPC features are free. You get Security Groups, Subnets, Route Tables, all kinds of shenanigans in a very stable way, with basically no incidents that I can remember. Except those are not free, they're just charged for separately under "egress transfer costs". End of the day, the engineers in the networking teams at AWS still have to get paid and that service probably also needs to run at a profit. When seen under this light, the costs make more sense to me than the usual simple view of "but bandwidth transfer costs are cheaper everywhere else!!!"
  - 7y ago
based2 7y ago
https://aws.amazon.com/en/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ https://aws.amazon.com/en/blogs/aws/amazon-eks-on-aws-fargat...
showerst 7y ago
Does anyone have experience with OVH's managed k8s offering? I've had good experiences with them in the past on pricing/quality.
- freedomben 7y ago
  I tried it out briefly and it seemed to work well. I never went to prod with it tho. I also didn't try out the LoadBalancer so I can't say how easy that would be to use. I've heard that cost can unexpectedly jump, so read the docs before you get too deep into it[1]: I now have an OpenShift cluster that I do testing with, but if I didn't I'd probably use OVH k8s in dev because it does seem by far the cheapest. [1] https://docs.ovh.com/gb/en/kubernetes/using-lb/ https://docs.ovh.com/gb/en/kubernetes/using-lb/
empath75 7y ago
As someone who manages a production cluster, I spend about 1% of my time worrying about the control plane. It’s trivial to get it running and keep it running now. It’s all the stuff you build on top of k8s that’s the hard part. I don’t see much value add to eks, personally.
- spalas 7y ago
  Do you use something like kops for setting up and maintaining your cluster? Most of my direct experience with Kubernetes has been on GKE, but I have been meaning to work through https://github.com/kelseyhightower/kubernetes-the-hard-way https://github.com/kelseyhightower/kubernetes-the-hard-way to gain more appreciation for what is going on behind the scenes.
Proven 7y ago
> For running fewer, larger clusters GKE is the most affordable option. For running that on GCP, perhaps. It may not be affordable for running managed K8S on-prem.
spicyramen 7y ago
Will be nice to include GPUs in v2 once there is more stable support for the operators
- spalas 7y ago
  Yeah -- Adding in GPUs and doing a deeper dive on how using some of the low-cost VM types (w/ small but burstable CPU, etc...) impact both cost & performance are things I hope to take a look at in the future!

168 points by spalas 7y ago

At the low end it’s worth considering Fargate distinct from EKS. You don’t need to provision a whole cluster (generally 3 machines minimum) and can just run as little as a single Pod.

petilon 7y ago
I tried Fargate and found it to be crappy. It is very hard to use. It is proprietary, so your app will not be portable, and your knowledge and experience will not be portable either. If you use Kubernetes there is tons of tutorials, your app becomes portable across clouds and your knowledge is portable from cloud to cloud too. GKE only costs around $60 per month for a single-machine "cluster".
- txcwpalpha 7y ago
  What is proprietary about Fargate? It's containers. I did not find any experience/knowledge (other than the basic knowledge of navigating the AWS console) that wouldn't transfer to any other container service.
  - petilon 7y ago
    AWS console is the crappy part. Azure and Google have much better GUIs. And here's the proprietary part: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/... For contrast, you can manage a Kubernetes deployment using standardized yaml and kubectl commands, regardless of whether the application is running on localhost (minikube), on Azure or on GKE. BTW, AWS Lightsail has decent GUI. Alas, it doesn't support containers out of the box. The best support for Docker image-based deployment is Azure App Service.
    - bdcravens 7y ago
      > here's the proprietary part: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/.. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/.... That's ECS, not EKS. Two different products. The EKS documentation is at https://docs.aws.amazon.com/eks/latest/userguide/fargate.html https://docs.aws.amazon.com/eks/latest/userguide/fargate.htm... > For contrast, you can manage a Kubernetes deployment using standardized yaml and kubectl commands, regardless of whether the application is running on localhost (minikube), on Azure or on GKE. Likewise for EKS.
      - petilon 7y ago
        I was replying to this: > At the low end it’s worth considering Fargate distinct from EKS.
        txcwpalpha 7y ago
        Right, and you linked to the documentation for ECS on Fargate rather than the documentation for Kubernetes Fargate, which is what was being talked about. Again, two different products.
    - txcwpalpha 7y ago
      I'm still not seeing the difference. As pointed out, what you linked is for ECS. That has nothing to do with Kubernetes, so I'm not sure why you're comparing the things on that page to kubectl commands on GKE or Azure. Of course you cannot use kubectl on ECS, because ECS has nothing to do with kube. When you are using actual EKS (with or without Fargate), you certainly can use standardized kubectl commands. The only "proprietary" things I see in your link is the specific AWS CLI commands used to set up the cluster before you can use kubectl, but both Azure and GCP require using the Azure CLI and gcloud CLI for cluster deployment, too. There's also setting up AWS-specific security groups and IAM roles, but you have to do those same things on GCP or Azure, too, and both of those have their own "proprietary" ways of setting up networking and security, so I don't see the differentiating factor.
- thoraway1010 7y ago
  I use fargate and pretty happy with it. Don't need big scale out - it supports $1M/year revenue so not huge, but LOVE the simplicity. I just have the CLI commands in my dockerfiles as comments, so once I get things sorted locally using docker I update the task with some copy / paste. I only update occasionally when I need to make some changes (locally do a lot more). The one thing I'd love to get my DOCKER image sizes down - they seem way too big for what they do but it's just easier to start with full fat images. I tried alpine images and couldn't get stuff to install / compile etc.
  - grey 7y ago
    You should look into multistage docker builds, that lets you still use a full fat image for your build but then leave all the build tools out of your final image I liked jpetazzo's post on the subject but there are plenty to choose from https://www.ardanlabs.com/blog/2020/02/docker-images-part1-reducing-image-size.html https://www.ardanlabs.com/blog/2020/02/docker-images-part1-r...
    - thoraway1010 7y ago
      Someone else suggested the same thing actually. Easy to get lazy when it "just works" and internet is 1gig home and office - you can see how bloat just builds up.
- scarface74 7y ago
  How is Fargate “proprietary”? It runs a Docker containers. But if you are at any type of scale, the last thing on your to do list is “cloud migration”.

0x1221 7y ago

I'm not familiar with the space so my question might not be that relevant - where does OpenShift fit in all of this (I still struggle to differentiate it from Kubernetes) and is there any merit to IBM trying to sell it so hard?

symfrog 7y ago
OpenShift is Kubernetes, just like RHEL is a Linux distribution with support for enterprises. OpenShift makes an opinionated choice about what they bundle (distribute) with vanilla Kubernetes. For example, Istio was chosen as the service mesh distributed with OpenShift 4.
- freedomben 7y ago
  I agree with you here, but do think it's worth pointing out that OpenShift adds some non-trivial features to Kubernetes and they aren't just plucked from existing projects and bundled. Some major K8s features started out as OpenShift features and got merged upstream (Ingress, Deployments, etc). Red Hat innovates and improves the distribution rather than just repackaging it and slapping on a support contract.
p_l 7y ago
Openshift wraps around Kubernetes, with some of their own special offering stuff on it. Generally, plain K8s is a building block - RedHat made Openshift with a bunch of opinionated choices, geared towards enterprise deployments, some of them migrating later to Kubernetes itself (OpenShift's Route inspired K8s' Ingress), some OpenShift cribs from K8s (Istio becoming part of OpenShift by default in OS 4). Generally OpenShift heavily targets enterprises as "All-in-One" package. Some of that works, some doesn't, but honestly it's often more a case of the IT dept that manages the install ;) Except installing OpenShift. That's horrific. Someone should repent for the install process, seriously.
- Conan_Kudo 7y ago
  Even with OpenShift 4? I thought it was pretty nice and straightforward, to be honest...

david-s 7y ago

It doesn't seem to include Digital Ocean in the comparison.

axaxs 7y ago
DO is my absolute favorite. I really think they could be a long term winner. Their interface is so much nicer than the competitors, in my opinion. I'm not even currently a customer, let alone a shill.
- steve_adams_86 7y ago
  I agree that DO is awesome. I'd argue though that they can make a better UI because they offer less. Everything is a littler simpler. It would be hard to condense AWS into a similar type of interface. Having said that, DO is enough for virtually everything I've ever worked on, and the user experience and price are so much better. They're a clear winner for almost everything I do these days.
  - axaxs 7y ago
    I agree but that's part of the charm to me. I only use what 4 or 5 things in AWS, but each login is information overload. Having to Ctrl F what you are looking for is not an ideal experience. Whether a conscious decision or not, I think offering what the 80ish percent (just a guess) actually use, and streamlining it, is the right decision.
  - freehunter 7y ago
    AWS could really use a dashboard where you can pick the components you want to see and only those show up. If I only use S3 and EC2 I shouldn’t have to search for those two products every time I log in.
    - scarface74 7y ago
      The console already shows the five most recently used components when you log in.

almostdigital 7y ago

Scaleway gives you a k8s cluster starting from 40 EUR per month

xbenjii 7y ago
They allow the DEV1-M boxes now, which are 8EUR a month.
neop1x 7y ago
Thanks for this! I've looked at Scaleway and so far I like it a lot, this looks like the end of my months of suffering. It seems to fit my needs perfectly. Finally a smaller cloud provider doing it right!

mcdoker18 7y ago

Only Azure doesn’t charge for the k8s control plane, that is the most surprising thing for me.

petilon 7y ago
We don't know how long that's going to last. Google didn't charge for control plane either, until recently. So Azure couldn't charge, in order to be competitive. Now that Google has started charging Azure may start too.
- fernandotakai 7y ago
  Azure also doesn't have an SLA, so that's why they don't charge. Google started charging after they added an SLA.
  - dodobirdlord 7y ago
    Makes sense, the point of an SLA is that you agree to pay back the customer's money if you don't meet it. What does it mean to have an SLA for a free product?

spectramax 7y ago

Just curious - what's wrong with buying a large instance (24 cores) and running it for < 10,000 users? Kubernetes feels like an insane complexity that doesn't need to be taken on and managed. You're gonna spend more time managing Kubernetes than writing actual software. Also, it feels like if something goes wrong in prod with your cluster - you're gonna need external help to get you back on the feet. If you're not going to build the next Facebook, why would you need so much complexity?

halfmatthalfcat 7y ago
Because it removes both provisioning and deployment concerns when you can build a container and then just tell Kubernetes to deploy it onto the cluster. Theres not much that goes into spinning up these managed Kuberentes clusters. Most of it is telling them what classes of instances you want created. When you buy a large instance, you still need to set up the instance and tweak it to your application's needs. You then need to babysit this node.
- warrenm 7y ago
  >When you buy a large instance, you still need to set up the instance and tweak it to your application's needs. You then need to babysit this node. It's called "use kickstart"
eknkc 7y ago
It is a single point of failure. I want multiple smaller instances even if I’m running something trivial. It is not about scale but about reliability for us. And the moment I go with multiple instances I need something to manage that mess. Kube handles it well.
- spectramax 7y ago
  I've used DigitalOcean's load balancer + multiple instances that run app containers. It works fine and without Kubernetes. I wasn't saying that just use 24 core single instance. Perhaps I should have worded it better.

madjam002 7y ago

Too bad AKS is just terrible. Slow provisioning time, slow PVCs, slow LoadBalancer provisioning, slow node pool management, plus non-production ready node pool implementation.

aliswe 7y ago
Agreed, not below usable though. Some more: rolling upgrades of k8s (said to not affect the uptime of the cluster) not being rolling in actuality, allowing upgrades when the service principal is expired thus preventing the nodes from being added to the LB, certain aks versions not being upgradable requiring you to recreate the cluster from scratch ...

neop1x 7y ago

Egress costs at those major clouds are ridiculous. Once you start having some traffic it can easily make 50% of all costs or more. That is not justifiable! Meanwhile hardware costs are going down. We need more k8s providers with more resonable pricing. Unfortunatelly both Digital Ocean and Oracle Cloud don't have proper network load balancer implementations which is a must for elastic regional clusters and to forward TCP in a way that client IP is preserved and be able to add nodes without downtimes or TCP resets. OVH cloud doesn't implement LoadBalancer service type at all. So the choice in 2020 is really just Google, Amazon, Azure with their rolls-royce pricing. The cost difference between them is neglible. And then there are confidential free credits for startups. So sad...

atombender 7y ago
For HTTP, an external load balancer/CDN such as CloudFlare or Fastly could easily fulfill that role, though you obviously have to jump through a few more hoops to get there, since it's not built in. But with some of them you also get things like TLS termination, DNS hosting, advanced rewriting, caching, and DDOS protection.
ikiris 7y ago
You seem to think theres some kind of disconnect between egress pricing, and if the network has a functional backbone with load balancing infrastrucutre. If anything, egress pricing is to low in the vast majority of cases, and it really shows when customers start using it.
vasco 7y ago
All of AWS's VPC features are free. You get Security Groups, Subnets, Route Tables, all kinds of shenanigans in a very stable way, with basically no incidents that I can remember. Except those are not free, they're just charged for separately under "egress transfer costs". End of the day, the engineers in the networking teams at AWS still have to get paid and that service probably also needs to run at a profit. When seen under this light, the costs make more sense to me than the usual simple view of "but bandwidth transfer costs are cheaper everywhere else!!!"
- 7y ago

based2 7y ago

https://aws.amazon.com/en/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/ https://aws.amazon.com/en/blogs/aws/amazon-eks-on-aws-fargat...

showerst 7y ago

Does anyone have experience with OVH's managed k8s offering? I've had good experiences with them in the past on pricing/quality.

freedomben 7y ago
I tried it out briefly and it seemed to work well. I never went to prod with it tho. I also didn't try out the LoadBalancer so I can't say how easy that would be to use. I've heard that cost can unexpectedly jump, so read the docs before you get too deep into it[1]: I now have an OpenShift cluster that I do testing with, but if I didn't I'd probably use OVH k8s in dev because it does seem by far the cheapest. [1] https://docs.ovh.com/gb/en/kubernetes/using-lb/ https://docs.ovh.com/gb/en/kubernetes/using-lb/

empath75 7y ago

As someone who manages a production cluster, I spend about 1% of my time worrying about the control plane. It’s trivial to get it running and keep it running now. It’s all the stuff you build on top of k8s that’s the hard part. I don’t see much value add to eks, personally.

spalas 7y ago
Do you use something like kops for setting up and maintaining your cluster? Most of my direct experience with Kubernetes has been on GKE, but I have been meaning to work through https://github.com/kelseyhightower/kubernetes-the-hard-way https://github.com/kelseyhightower/kubernetes-the-hard-way to gain more appreciation for what is going on behind the scenes.

Proven 7y ago

> For running fewer, larger clusters GKE is the most affordable option. For running that on GCP, perhaps. It may not be affordable for running managed K8S on-prem.

spicyramen 7y ago

Will be nice to include GPUs in v2 once there is more stable support for the operators

spalas 7y ago
Yeah -- Adding in GPUs and doing a deeper dive on how using some of the low-cost VM types (w/ small but burstable CPU, etc...) impact both cost & performance are things I hope to take a look at in the future!