4 ms·
This is a great article but leaves out a key idea. Load shedding is really a key topic in cost utilization. You need load shedding to be able to serve closer to
by jeffbee 5y ago
This is a great article but leaves out a key idea. Load shedding is really a key topic in cost utilization. You need load shedding to be able to serve closer to your capacity red line. You can always buy your way out of overload. Load shedding is a feature that, if you have it, allows you to more comfortably dial back your resources and serve closer to the limit.
- ozzythecat 5y ago> You can always buy your way out of overload. Not really. My service may depend on other services that I have no control over. Perhaps I have extra money to scale up my own service, but those other services may be owned by different teams or organizations entirely.
- shukantpal 5y agoWith money, you can in-house those services and scale them up if those "other" organizations won't scale up.
- ozzythecat 5y agoWith infinite money, sure. Realistically you won’t have the budget. And even if it’s “in house”, it’s going to be some service run by another team or organization within your company that has their own priorities, roadmap, and deliverables. They’re not just going to scale up their service because you asked them. They basically don’t have to do anything for you at all. And even if they agree - everything has a cost. There’s simply the overhead of even sending an email or a Slack message and setting up a meeting, getting people ramped up on your use case, why you need them to scale, for how long, for what use cases, etc etc. everything has a cost and money and budget are always constrained.
- shukantpal 5y agoThat's why the "with money" not "with little money"
- dyanacek 5y agoAgreed - utilization is an important consideration here. The capacity red line will still be there, but when load shedding is effective, the impact of crossing that red line is les. It’d be an error rate linearly proportionally to the excess, rather than the service falling off a cliff. But for the services this is talking about, neither case is okay, so we put a ton of emphasis on auto scaling models to make sure we don’t get into the situation. A key sort of “continuation” to this article is the one on fairness: https://aws.amazon.com/builders-library/fairness-in-multi-tenant-systems/ https://aws.amazon.com/builders-library/fairness-in-multi-te... . This gets into the topic of utilization a bit more. But you’re right - good load shedding gives a business a tool to make an easier trade off when it comes to capacity management. A slight error rate until autoscaling kicks in is an easier pill to swallow than a worse outage.