3 ms·

... unless the use case is weird, e.g. you need massive amounts of processing power infrequently.

vidarh 8y ago
There certainly are people with use-cases like that, but it is rare to find people who both have large, sudden spikes that are not better handled in other ways (e.g. combination of CDNs and micro-caching), or where the load can not be trivially smoothed out enough (e.g. there's enough work with relaxed latency requirements or that can be scheduled to off peak times, or where the spikes are not small enough relative to base-load that it still pays off to put most load on dedicated equipment and use AWS or similar to handle spikes only. People often think they have workloads like that. I've done work on plenty of setups over the years were people were certain they had lots of big spikes, but most of them only had moderate day/night cycles (and usually a lot less spiky than they thought, because they'd never bothered to measure). Only in a few cases I've seen usage patterns that justified spinning instances up/down very often, and for those cases, ironically the availability of clouds have made dedicated hosting more cost effective, because if you host everything in VMs or containers anyway, you can now get to far higher utilization levels on your on-prem/colocated setups, and be prepared to spin up some cloud instances (automatically, if you wish, it's not that much effort), and tie them into your cluster when your load warrants it. It's not suitable for all workloads, but it works surprisingly well if anything. I do see that argument a lot for using cloud providers, but the vast majority of cloud setups I've seen end up having very static workloads growing at rates that gives plenty of time to prepare.
- icelancer 8y ago
  > People often think they have workloads like that. This is exactly what I see on the regular. The AWS model promises huge savings if you have these cyclical use cases and rarely does this ever happen in a small business or startup.
  - devonkim 8y ago
    Worse, you can have such needs but wind up overprovisioning because your software is tough to autoscale automatically up or down. In that case you should probably pay for dedicated, mostly static servers in traditional setups until you can’t stomach the costs of being overprovisioned anymore then go to a cloud that lets you autoscale to recover a lot of your costs. Sure, reserved instances help substantially but because it’s a cloud instances themselves can go up or down without the best SLAs (I still say I’ve had more physical servers among the tens of thousands seen go belly up than the hundreds of thousands of EC2 instances I’ve seen in my experience). Other factors are obviously the ability to get a compute instance up immediately or because you have bad operational situations (horrible legacy DC equipment, genuinely bad admins or vendors, etc) that far outweigh the opex and capex parts of the equation and bleed into SLA, poor reputation / relationships, etc.
  - nijave 8y ago
    This does apply to non prod environments. Development/test environments that have low utilization and only run during business hours are better candidates
Nursie 8y ago
> ... unless the use case is weird, e.g. you need massive amounts of processing power infrequently I worked for a large organisation that had this need. It would require ~10K cores and a large number of GPUs, for a period of a few days, approximately once a month (or less). That sort of ask appears to be too much for amazon, and they requested for us to pre-book the capacity for the whole month, wiping out any cost savings we might have made over just buying the hardware.

There certainly are people with use-cases like that, but it is rare to find people who both have large, sudden spikes that are not better handled in other ways (e.g. combination of CDNs and micro-caching), or where the load can not be trivially smoothed out enough (e.g. there's enough work with relaxed latency requirements or that can be scheduled to off peak times, or where the spikes are not small enough relative to base-load that it still pays off to put most load on dedicated equipment and use AWS or similar to handle spikes only. People often think they have workloads like that. I've done work on plenty of setups over the years were people were certain they had lots of big spikes, but most of them only had moderate day/night cycles (and usually a lot less spiky than they thought, because they'd never bothered to measure). Only in a few cases I've seen usage patterns that justified spinning instances up/down very often, and for those cases, ironically the availability of clouds have made dedicated hosting more cost effective, because if you host everything in VMs or containers anyway, you can now get to far higher utilization levels on your on-prem/colocated setups, and be prepared to spin up some cloud instances (automatically, if you wish, it's not that much effort), and tie them into your cluster when your load warrants it. It's not suitable for all workloads, but it works surprisingly well if anything. I do see that argument a lot for using cloud providers, but the vast majority of cloud setups I've seen end up having very static workloads growing at rates that gives plenty of time to prepare.

icelancer 8y ago
> People often think they have workloads like that. This is exactly what I see on the regular. The AWS model promises huge savings if you have these cyclical use cases and rarely does this ever happen in a small business or startup.
- devonkim 8y ago
  Worse, you can have such needs but wind up overprovisioning because your software is tough to autoscale automatically up or down. In that case you should probably pay for dedicated, mostly static servers in traditional setups until you can’t stomach the costs of being overprovisioned anymore then go to a cloud that lets you autoscale to recover a lot of your costs. Sure, reserved instances help substantially but because it’s a cloud instances themselves can go up or down without the best SLAs (I still say I’ve had more physical servers among the tens of thousands seen go belly up than the hundreds of thousands of EC2 instances I’ve seen in my experience). Other factors are obviously the ability to get a compute instance up immediately or because you have bad operational situations (horrible legacy DC equipment, genuinely bad admins or vendors, etc) that far outweigh the opex and capex parts of the equation and bleed into SLA, poor reputation / relationships, etc.
- nijave 8y ago
  This does apply to non prod environments. Development/test environments that have low utilization and only run during business hours are better candidates

Nursie 8y ago

> ... unless the use case is weird, e.g. you need massive amounts of processing power infrequently I worked for a large organisation that had this need. It would require ~10K cores and a large number of GPUs, for a period of a few days, approximately once a month (or less). That sort of ask appears to be too much for amazon, and they requested for us to pre-book the capacity for the whole month, wiping out any cost savings we might have made over just buying the hardware.