4 ms·

Hey Seth, Thanks for being the recipient of everyone's (justifiable) frustrations. They probably don't pay you enough. I think, what is especially frustrating

Hey Seth, Thanks for being the recipient of everyone's (justifiable) frustrations. They probably don't pay you enough. I think, what is especially frustrating about this, is that we do already pay for resources that are provisioned by our K8S clusters. We pay for the network traffic, the storage, the compute. I saw you mention StackDriver... we pay for that as well. I can appreciate that actually setting up and managing GKE backplanes is a non-trivial expense, but I generally assumed that that cost was amortized out, just like I don't pay for the backplane that runs GCE and the rest of GCP's service suite. I also appreciate that you mention some customers are perhaps taking advantage of this "free" resource. But, isn't that quotas are for? Frankly, more concerning than the fact that now I have a new $73/mo. fee attached to my account (which, is not the end of the world) is that this really comes out of left field, and in the context of concerns about the the nature of GCP's new leadership, and reports of Google leadership debating GCP as a going concern. I realize a lot of that isn't well founded, but it's surprises like this one that keep that narrative alive. AWS ain't no saint, but they are pretty consistently who they are: not full of bad surprises. This just leaves a bad taste in the mouth, and makes me wonder if I can expect other surprising cost increases, or perhaps, if these don't "work", worse surprises like deprecation notices. Is this the precursor to you all discontinuing GKE because, as the DevRel class likes to tweet, nobody should be using Kubernetes if they can use (more expensive) services like Cloud Run? Are we about to get Oracled?

sethvargo 7y ago
> They probably don't pay you enough. Can confirm :) > ...we do already pay for resources that are provisioned by our K8S clusters Customers are charged for worker nodes, but until this point, the control plane ("master") nodes have been free. In addition to the raw compute costs for those nodes, there's the SRE overhead for managing, upgrading, and securing them. > ...but I generally assumed that that cost was amortized out <googlehat>I'm not really sure.</googlehat> <civilian>My guess would be that, initially, this was the case. However, over time, people have created many zero-node clusters. Now the amortization isn't. Again, pure speculation.</civilian> > But, isn't that quotas are for? See my comment above about zero-node clusters. > I have a new $73/mo. fee attached to my account (which, is not the end of the world) is that this really comes out of left field... Acknowledge, but I do want to highlight that changes take place a few months from now (June 2020), not immediately. Furthermore, each billing account gets one zonal cluster with no management fee. > Is this the precursor to you all discontinuing GKE because, as the DevRel class likes to tweet, nobody should be using Kubernetes if they can use (more expensive) services like Cloud Run? 100% no. Also, Cloud Run is almost always cheaper than running a Kubernetes cluster. > Are we about to get Oracled? I'm not sure what you mean by that verb.
- numbsafari 7y ago
  Thank you for the detailed response. > In addition to the raw compute costs for those nodes, there's the SRE overhead for managing, upgrading, and securing them. By that logic, can we expect to see charges for GCP Projects and the GCP Console? Cloud IAM? > people have created many zero-node clusters I'd be really curious what is driving folks to do that. Are they using the backplane for CRDs and custom controllers and no compute? This feels like it could be addressed similar to alpha clusters, or with a quota, e.g.: clusters with 0 nodes for > 24 hours will be terminated? Separately, It seems like handing everyone 3 months to figure out what to do about a new $73 * X fee isn't the best plan. Including some kind of estimate in the emails that were sent out would have been helpful. There was a change in pricing for StackDriver a while back that did this. It was very helpful to understand how we would be impacted. > Furthermore, each billing account gets one zonal cluster with no management fee. My feedback is that you would probably get getting way less blowback if that free-tier didn't come across as inadequate. I can appreciate that there are use-cases where it makes sense for you all to be charging. But one zonal cluster... It makes the whole thing feel punitive. > I'm not sure what you mean by that verb. I have a feeling we're all about to go on a journey of discovery together.
  - ssmw 7y ago
    If abuse of zero-node clusters is an issue, wouldn't it be better to introduce a zero-node cluster fee the same way you charge for unused reserved IP addresses?
    - remram 7y ago
      Also, how much resource does a 0-node cluster actually use on the control plane?
  - jonas21 7y ago
    > I'd be really curious what is driving folks to do that. I was one of those people. I got an email from Google this morning and thought "that's weird. I didn't even know I was running a Kubernetes cluster." I think I created it years ago to work through a Kubernetes tutorial and, since it was free, never bothered to delete it. So, I can imagine this being a problem. Though it seems like having a minimum hourly charge per cluster would have been a better way to handle this (i.e. if your cluster is using less than $0.10/hr in resources, you get charged the difference).

sethvargo 7y ago

> They probably don't pay you enough. Can confirm :) > ...we do already pay for resources that are provisioned by our K8S clusters Customers are charged for worker nodes, but until this point, the control plane ("master") nodes have been free. In addition to the raw compute costs for those nodes, there's the SRE overhead for managing, upgrading, and securing them. > ...but I generally assumed that that cost was amortized out <googlehat>I'm not really sure.</googlehat> <civilian>My guess would be that, initially, this was the case. However, over time, people have created many zero-node clusters. Now the amortization isn't. Again, pure speculation.</civilian> > But, isn't that quotas are for? See my comment above about zero-node clusters. > I have a new $73/mo. fee attached to my account (which, is not the end of the world) is that this really comes out of left field... Acknowledge, but I do want to highlight that changes take place a few months from now (June 2020), not immediately. Furthermore, each billing account gets one zonal cluster with no management fee. > Is this the precursor to you all discontinuing GKE because, as the DevRel class likes to tweet, nobody should be using Kubernetes if they can use (more expensive) services like Cloud Run? 100% no. Also, Cloud Run is almost always cheaper than running a Kubernetes cluster. > Are we about to get Oracled? I'm not sure what you mean by that verb.

numbsafari 7y ago
Thank you for the detailed response. > In addition to the raw compute costs for those nodes, there's the SRE overhead for managing, upgrading, and securing them. By that logic, can we expect to see charges for GCP Projects and the GCP Console? Cloud IAM? > people have created many zero-node clusters I'd be really curious what is driving folks to do that. Are they using the backplane for CRDs and custom controllers and no compute? This feels like it could be addressed similar to alpha clusters, or with a quota, e.g.: clusters with 0 nodes for > 24 hours will be terminated? Separately, It seems like handing everyone 3 months to figure out what to do about a new $73 * X fee isn't the best plan. Including some kind of estimate in the emails that were sent out would have been helpful. There was a change in pricing for StackDriver a while back that did this. It was very helpful to understand how we would be impacted. > Furthermore, each billing account gets one zonal cluster with no management fee. My feedback is that you would probably get getting way less blowback if that free-tier didn't come across as inadequate. I can appreciate that there are use-cases where it makes sense for you all to be charging. But one zonal cluster... It makes the whole thing feel punitive. > I'm not sure what you mean by that verb. I have a feeling we're all about to go on a journey of discovery together.
- ssmw 7y ago
  If abuse of zero-node clusters is an issue, wouldn't it be better to introduce a zero-node cluster fee the same way you charge for unused reserved IP addresses?
  - remram 7y ago
    Also, how much resource does a 0-node cluster actually use on the control plane?
- jonas21 7y ago
  > I'd be really curious what is driving folks to do that. I was one of those people. I got an email from Google this morning and thought "that's weird. I didn't even know I was running a Kubernetes cluster." I think I created it years ago to work through a Kubernetes tutorial and, since it was free, never bothered to delete it. So, I can imagine this being a problem. Though it seems like having a minimum hourly charge per cluster would have been a better way to handle this (i.e. if your cluster is using less than $0.10/hr in resources, you get charged the difference).