6 ms·

HA seems like a poor choice as a default. I don't want to pay ~$150/month for dev and staging clusters, and I certainly don't need them to be HA. IIUC, Google/

HA seems like a poor choice as a default. I don't want to pay ~$150/month for dev and staging clusters, and I certainly don't need them to be HA. IIUC, Google/GKE and Microsoft/AKS are single-zone by default... free for masters, but not free for HA.

user5994461 8y ago
Google and Amazon are enterprise services. If you consider $150 too expensive, you are not part of the target customers.
- yebyen 8y ago
  The Google GKE offering can operate with a single g1-small worker. That's $0.0257/hr or $0.0070/hr preemptible. $5/mo minimum* This is not even the same ballpark. *(to be fair, the $5/mo minimum does not include a private network and a load balancer, and you will almost definitely need both in order to actually do anything with a cluster.)
  - nat 8y ago
    You're thinking like a human. There are plenty of organizations where $5/mo and $150/mo are essentially indistinguishable and would not affect decision making in any way.
    - yebyen 8y ago
      That's $150/mo before you even launch a single worker tho. I don't know about you, but I'd personally rather pay the $5 if I don't need a serious cluster yet. How much in actual absolute dollars do you think this is going to cost you, just in order to figure out exactly how it works, and what other AWS widgets you'll also need to pay for in order to use it effectively? I am a human, basically looking for a stepping stone on some actual cloud to land on that is comparable to Minikube, and this sure isn't it.
      - deleted 8y ago
        [deleted]
      - yebyen 8y ago
        I'm pretty clearly in the minority with my opinion here, and I recognize that. Amazon has made a substantial investment in Kubernetes here and that is nothing to shrug at. They are entitled (and justified) to recoup it. It's fair to say that K8S is not for lightweights, too. AWS provides a fairly generous free tier and they are also justified to extract some payment from every user of their services that derives some non-negligible value. If you're using Kubernetes and you're not getting any value out of it, I think it's also fair to say that you're doing it wrong.
        user5994461 8y ago
        You're not in the minority. It's well understood that there are amateurs trying to use the expensive clouds and shave off every penny, while the main customers are enterprise customers who couldn't notice a thousand dollars more or less. It's just pointless for small users to argue about AWS costs. The service is not meant for them and it cannot fulfill their requirements.
        yebyen 8y ago
        You're obviously no slouch, and I respect your opinion, but I think Amazon is all about the long tail and without speculating too much about what they are thinking, I'll say they might disagree. They are obviously smart to go for the big money dollars first, but everything I know about "Long Tail" says you're wrong. I absolutely agree with everything you said, if you only added "at this time" to the end of that last sentence.
        user5994461 8y ago
        What long tail do you mean? I would say the small tech companies with tens of developers and tens of servers are the long tail for AWS. The service is too expensive and too complicated to be used by very small companies or individual users. There are also more appropriate competitors in that space.
    - derefr 8y ago
      And plenty of startups that choose between AWS and GCP at a point when those numbers are plenty distinguishable, even though later on they'll become those organizations.
      - user5994461 8y ago
        And they will all die pretty quickly for running their infrastructure on a pre emptible instance to save a few dollars. By the way, startups get $100k free credits on each of the major clouds.
        yebyen 8y ago
        You can afford ~5 preemptible instances for the cost of one full-time instance. I think you are severely overestimating the harm that is done by preempting/terminating instances in a prod Kubernetes cluster that is capably configured for full HA and distributed across zones. That's what it's made for – nodes are as disposable as pods. They may come and go as they please, and they'll be replaced as needed by the self-healing nature of the cluster and the scaling group. (Why save a few dollars when you can get extra capacity instead? Especially when you have $100k of someone else's money to burn... so, for real though, anyone who has actually tried this configuration can chime in and confirm that it works as well as I imagine.) I'm pretty sure that periodically shutting down some nodes is a boon for cluster utilization, too. One of the things that Kubernetes does not do on its own, is load rebalancing. You can configure the autoscaler to recognize when nodes are overprovisioned, and let it drain a few and shut them down... or you can let the preemptible nature of (some/all of your) nodes do it for you. (Why not both? Some nodes are getting killed, or you're paying for resources you don't use, so... one way or another.)
        user5994461 8y ago
        Your message obviously implies that you have no experience with what you are suggesting, hence why you don't realize the downsides.
        8y ago
    - foxylion 8y ago
      I second this. This means you don't have to setup and manage those master nodes. AWS will do this for you. A developer fixing an issue with your self-managed master nodes will be much more expensive than 150$/month. As an individual this may look expensive, but as a company this is much cheaper than paying the loan of a developer.
      - jpatokal 8y ago
        I suspect you missed the point earlier: GKE offers a free managed master, even if your cluster is a single node costing you $5/month. Disclaimer: I work at GCP, but not on GKE.
        foxylion 8y ago
        Yes it is nice for experiments, but if you already invest thousands of dollars a month for cloud resources, an additional 150$ are for sure not a problem.
    - threeseed 8y ago
      Exactly. We spend hundreds of thousands a month with AWS. $144/month is nothing if it means someone else can manage some of the parts of Kubernetes.
  - cube2222 8y ago
    You can set up the nginx controller with NodePorts to get rid of the load balancer if it's just a side-project cluster. (Otherwise load balancers will be the majority of your cost)
    - yebyen 8y ago
      Thanks for that!
thesandlord 8y ago
GKE is free for HA as well: https://cloudplatform.googleblog.com/2018/06/Regional-clusters-in-Google-Kubernetes-Engine-are-now-generally-available.html https://cloudplatform.googleblog.com/2018/06/Regional-cluste...
exelius 8y ago
Looks like they want companies to pay to avoid the platform lock-in that k8s helps you avoid...
idunno246 8y ago
Until recently gke was charging 0.15/ hour too, so aws is just a little behind https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster-Management-Fees-on-Google-Kubernetes-Engine.html?m=1 https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster...
- yebyen 8y ago
  Not only recently, but actually this was the cost until yesterday for GKE with HA masters distributed across AZs[1]! I only heard that this announcement from AWS was likely coming this morning, over here in the thread about upgrading GKE clusters[2]. Given how long we've waited since the announcement at November's re:Invent, there's honestly not a lot that seems terribly rushed about this news, but I bet that AWS really would have liked to have Ingress controllers that are integrated with ALB ready for this announcement to go with their CNI plugins. "Why am I waiting for Amazon to get out of preview when Google's been giving it away this whole time?" For AWS customers that aren't locked in, that seems like a reasonable train of thought. [1]: https://cloudplatform.googleblog.com/2018/06/Regional-clusters-in-Google-Kubernetes-Engine-are-now-generally-available.html https://cloudplatform.googleblog.com/2018/06/Regional-cluste... [2]: https://news.ycombinator.com/item?id=17236794 https://news.ycombinator.com/item?id=17236794

user5994461 8y ago

Google and Amazon are enterprise services. If you consider $150 too expensive, you are not part of the target customers.

yebyen 8y ago
The Google GKE offering can operate with a single g1-small worker. That's $0.0257/hr or $0.0070/hr preemptible. $5/mo minimum* This is not even the same ballpark. *(to be fair, the $5/mo minimum does not include a private network and a load balancer, and you will almost definitely need both in order to actually do anything with a cluster.)
- nat 8y ago
  You're thinking like a human. There are plenty of organizations where $5/mo and $150/mo are essentially indistinguishable and would not affect decision making in any way.
  - yebyen 8y ago
    That's $150/mo before you even launch a single worker tho. I don't know about you, but I'd personally rather pay the $5 if I don't need a serious cluster yet. How much in actual absolute dollars do you think this is going to cost you, just in order to figure out exactly how it works, and what other AWS widgets you'll also need to pay for in order to use it effectively? I am a human, basically looking for a stepping stone on some actual cloud to land on that is comparable to Minikube, and this sure isn't it.
    - deleted 8y ago
      [deleted]
    - yebyen 8y ago
      I'm pretty clearly in the minority with my opinion here, and I recognize that. Amazon has made a substantial investment in Kubernetes here and that is nothing to shrug at. They are entitled (and justified) to recoup it. It's fair to say that K8S is not for lightweights, too. AWS provides a fairly generous free tier and they are also justified to extract some payment from every user of their services that derives some non-negligible value. If you're using Kubernetes and you're not getting any value out of it, I think it's also fair to say that you're doing it wrong.
      - user5994461 8y ago
        You're not in the minority. It's well understood that there are amateurs trying to use the expensive clouds and shave off every penny, while the main customers are enterprise customers who couldn't notice a thousand dollars more or less. It's just pointless for small users to argue about AWS costs. The service is not meant for them and it cannot fulfill their requirements.
        yebyen 8y ago
        You're obviously no slouch, and I respect your opinion, but I think Amazon is all about the long tail and without speculating too much about what they are thinking, I'll say they might disagree. They are obviously smart to go for the big money dollars first, but everything I know about "Long Tail" says you're wrong. I absolutely agree with everything you said, if you only added "at this time" to the end of that last sentence.
        user5994461 8y ago
        What long tail do you mean? I would say the small tech companies with tens of developers and tens of servers are the long tail for AWS. The service is too expensive and too complicated to be used by very small companies or individual users. There are also more appropriate competitors in that space.
  - derefr 8y ago
    And plenty of startups that choose between AWS and GCP at a point when those numbers are plenty distinguishable, even though later on they'll become those organizations.
    - user5994461 8y ago
      And they will all die pretty quickly for running their infrastructure on a pre emptible instance to save a few dollars. By the way, startups get $100k free credits on each of the major clouds.
      - yebyen 8y ago
        You can afford ~5 preemptible instances for the cost of one full-time instance. I think you are severely overestimating the harm that is done by preempting/terminating instances in a prod Kubernetes cluster that is capably configured for full HA and distributed across zones. That's what it's made for – nodes are as disposable as pods. They may come and go as they please, and they'll be replaced as needed by the self-healing nature of the cluster and the scaling group. (Why save a few dollars when you can get extra capacity instead? Especially when you have $100k of someone else's money to burn... so, for real though, anyone who has actually tried this configuration can chime in and confirm that it works as well as I imagine.) I'm pretty sure that periodically shutting down some nodes is a boon for cluster utilization, too. One of the things that Kubernetes does not do on its own, is load rebalancing. You can configure the autoscaler to recognize when nodes are overprovisioned, and let it drain a few and shut them down... or you can let the preemptible nature of (some/all of your) nodes do it for you. (Why not both? Some nodes are getting killed, or you're paying for resources you don't use, so... one way or another.)
        user5994461 8y ago
        Your message obviously implies that you have no experience with what you are suggesting, hence why you don't realize the downsides.
        8y ago
  - foxylion 8y ago
    I second this. This means you don't have to setup and manage those master nodes. AWS will do this for you. A developer fixing an issue with your self-managed master nodes will be much more expensive than 150$/month. As an individual this may look expensive, but as a company this is much cheaper than paying the loan of a developer.
    - jpatokal 8y ago
      I suspect you missed the point earlier: GKE offers a free managed master, even if your cluster is a single node costing you $5/month. Disclaimer: I work at GCP, but not on GKE.
      - foxylion 8y ago
        Yes it is nice for experiments, but if you already invest thousands of dollars a month for cloud resources, an additional 150$ are for sure not a problem.
  - threeseed 8y ago
    Exactly. We spend hundreds of thousands a month with AWS. $144/month is nothing if it means someone else can manage some of the parts of Kubernetes.
- cube2222 8y ago
  You can set up the nginx controller with NodePorts to get rid of the load balancer if it's just a side-project cluster. (Otherwise load balancers will be the majority of your cost)
  - yebyen 8y ago
    Thanks for that!

thesandlord 8y ago

GKE is free for HA as well: https://cloudplatform.googleblog.com/2018/06/Regional-clusters-in-Google-Kubernetes-Engine-are-now-generally-available.html https://cloudplatform.googleblog.com/2018/06/Regional-cluste...

exelius 8y ago

Looks like they want companies to pay to avoid the platform lock-in that k8s helps you avoid...

idunno246 8y ago

Until recently gke was charging 0.15/ hour too, so aws is just a little behind https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster-Management-Fees-on-Google-Kubernetes-Engine.html?m=1 https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster...

yebyen 8y ago
Not only recently, but actually this was the cost until yesterday for GKE with HA masters distributed across AZs[1]! I only heard that this announcement from AWS was likely coming this morning, over here in the thread about upgrading GKE clusters[2]. Given how long we've waited since the announcement at November's re:Invent, there's honestly not a lot that seems terribly rushed about this news, but I bet that AWS really would have liked to have Ingress controllers that are integrated with ALB ready for this announcement to go with their CNI plugins. "Why am I waiting for Amazon to get out of preview when Google's been giving it away this whole time?" For AWS customers that aren't locked in, that seems like a reasonable train of thought. [1]: https://cloudplatform.googleblog.com/2018/06/Regional-clusters-in-Google-Kubernetes-Engine-are-now-generally-available.html https://cloudplatform.googleblog.com/2018/06/Regional-cluste... [2]: https://news.ycombinator.com/item?id=17236794 https://news.ycombinator.com/item?id=17236794