3 ms·

You don't have the individual vms check in. You have the VM coordinator report how many vms are running and get back an affirmation that it can cache until the

You don't have the individual vms check in. You have the VM coordinator report how many vms are running and get back an affirmation that it can cache until the next reporting period that the total is not over budget. If over budget, coordinator begins halting services. API gateways are similarly sending metrics somewhere. The coordinator can be the place to ingest that data and send the aggregated info to billing. If it gets back over budget, start halting endpoints. etc. Or do it within the billing service, but fire off a shutdown notification to the coordinator of whatever service created a billing record if over budget. Same idea. Basically, batch, amortize and cache work. Same as every computer problem. You establish some SLO for how much time your services can continue running after an overage has occurred, and if that's a couple minutes or whatever that will cut out like 99.99% of the impact in these stories.

danpalmer 2y ago
Solving this for any one resource type, or one billing axis, is absolutely achievable in the ways you've suggested. Solving this across all resource types and billing axes however is a different problem. You can't cache the notion than a VM is under the billing cap for an hour if there's another service that push spend over the cap within that hour. You're right that you could establish SLOs across everything and minimise the amount of monetary loss, in theory, but at scale (as some resource types necessarily bill infrequently, as customers are spending more per hour), I suspect even this breaks down. Then there's still the issue of billing at rest. Do you shut off VMs? That might be an easy question to answer. Do you delete their storage though? Harder. Do you delete blob storage? Probably not, but you've got to swallow the cost of that until the customer decides to pay again.

danpalmer 2y ago

Solving this for any one resource type, or one billing axis, is absolutely achievable in the ways you've suggested. Solving this across all resource types and billing axes however is a different problem. You can't cache the notion than a VM is under the billing cap for an hour if there's another service that push spend over the cap within that hour. You're right that you could establish SLOs across everything and minimise the amount of monetary loss, in theory, but at scale (as some resource types necessarily bill infrequently, as customers are spending more per hour), I suspect even this breaks down. Then there's still the issue of billing at rest. Do you shut off VMs? That might be an easy question to answer. Do you delete their storage though? Harder. Do you delete blob storage? Probably not, but you've got to swallow the cost of that until the customer decides to pay again.