4 ms·
Are there any protections these days at the cloud provider level? Like setting a maximum budget for a certain service (EC2, Aurora?) because downtime is prefer
by EGreg 1y ago
Are there any protections these days at the cloud provider level?
Like setting a maximum budget for a certain service (EC2, Aurora?) because downtime is preferable to this?
- bc569a80a344f9c 1y agoNot _really_. AWS has a budget tool, but it doesn’t natively support shutting down services. Of course, you can ingest the alerts it sends any way you want, including feeding them into pipelines that disable services. There’s plenty of blueprints you can copy for this. More seriously - and this is a legitimate technical limitation - of course AWS doesn’t check each S3 request or Lambda invocation against your budget, instead, it consolidates periodically via background reporting processes. That means there’s some lag, and you are responsible for any costs incurred that go over budget between such reporting runs.
- chuckadams 1y ago> of course AWS doesn’t check each S3 request or Lambda invocation against your budget If it can bill them per-invocation, why can't it also check against a budget? I don't expect it to be synchronous, but a lag of minutes to respond is still better than nothing. Can you even opt-in to shutting down services from the budget tool, or is that still something you have to script by hand from Cloudwatch alarms?
- bc569a80a344f9c 1y agoYou script it by hand. I think figuring out how to do this faster is less trivial than it might sound. I agree that synchronous checks aren’t reasonable. But let’s take Lambdas. They can run for 15 minutes, and if you consolidate within five minutes after a resource has been billed, that gives you a twenty minute lag. I’m not trying to make apologies for Amazon, mind you. Just saying that this isn’t exactly easy at scale, either. Sure, they bill by invocation, but that’s far from synchronous, too. In fact, getting alerts might very well be happening at the frequency of billing reconciliation, which might be an entirely reasonable thing to do. You could then argue that that process should happen more frequently, at Amazon’s cost.
- McGlockenshire 1y ago> but it doesn’t natively support shutting down services [...] of course AWS doesn’t check each S3 request or Lambda invocation against your budget, instead, it consolidates periodically via background reporting processes So, in other words, the vendor has provided substandard tooling with the explicit intent of forcing you to spend more money.
- franktankbank 1y agoJust set alerts that are not really timely and homeroll your own kill scripts its easy. It doesn't really work but its not really any harder than just fucking self hosting.
- JJMcJ 1y agoThat's why I like VPS setups. You hit the monthly maximum, and it just stops working. I host demos, not running a business, so it's less of an issue to get interrupted. Better an interruption than a $50,000 bill for forgetting to shut off a test database from last Wednesday.
- qcnguy 1y agoThat's not comparable. With a VPS there is no monthly maximum, just a max load on a second by second basis. You can be hit with traffic of which 90% bounces because your server is down, get nowhere near your intended monthly maximum, and then the rest of the month is quiet.
- McGlockenshire 1y agoYou seem to be describing this as a bad thing instead of the objectively good thing that it is.
- JJMcJ 1y agoI got VPS and other types of shared hosting mixed up, though I have seen VPS offerings with monthly or daily maximums.
- emseetech 1y agoUnless a startup has five+ nines service contracts with their customers already, a little bit of downtime once in a while is not the end of the world the cloud services want us to believe.