3 ms·

Given how often I've seen > There is no runtime available right now. Please change the compute type or try again later. recently, that's not always true. It'

Given how often I've seen > There is no runtime available right now. Please change the compute type or try again later. recently, that's not always true. It's always funny to me when you run into a wall of "no more instances" in AWS. It doesn't happen often unless you're at a massive scale, but it does happen surprisingly often with the GPU types

londons_explore 4y ago
Happens with Google cloud too, sometimes for weeks at a time. It's pretty sad how Google/Amazon can't at least temporarily move their own workloads to free up resources for their public clouds while they install new hardware.
- H8crilA 4y ago
  > free up resources (...) while they install new hardware That's likely not the reason. It's more likely caused by inherent spike-y ness of the traffic, given that most GPU instances are probably used for batch numerical computations (like training neural networks). The pricing incentivizes using a lot of nodes for a shorter amount of time, much like in the SHA example.
  - londons_explore 4y ago
    The pricing incentivises spot nodes, which use dynamic pricing to try to prevent this...
- dspillett 4y ago
  They are never going to significantly prioritise “public” cloud resource over internal needs, beyond own needs having quotas to make sure they don't run amok if something gets unexpectedly out of shape. If their own public services slowed down in high demand times, it wouldn't look good. The sales channels could try to spin it as “giving you priority over us” but who would genuinely believe them. They could reduce resources available to low priority internal stuff during spikes of other activity, but that would not have a large effect considering the combined size of activity on the public cloud resources.
ramraj07 4y ago
Is it surprising that relatively niche configurations (especially ones which clearly need specific hardware per compute) are finite in aws? But imagine aws running out of m4.X-Large or something. That would be interesting. What scale would you need to run out of the most common instance in a top availability zone? 10k? 100k?

londons_explore 4y ago

Happens with Google cloud too, sometimes for weeks at a time. It's pretty sad how Google/Amazon can't at least temporarily move their own workloads to free up resources for their public clouds while they install new hardware.

H8crilA 4y ago
> free up resources (...) while they install new hardware That's likely not the reason. It's more likely caused by inherent spike-y ness of the traffic, given that most GPU instances are probably used for batch numerical computations (like training neural networks). The pricing incentivizes using a lot of nodes for a shorter amount of time, much like in the SHA example.
- londons_explore 4y ago
  The pricing incentivises spot nodes, which use dynamic pricing to try to prevent this...
dspillett 4y ago
They are never going to significantly prioritise “public” cloud resource over internal needs, beyond own needs having quotas to make sure they don't run amok if something gets unexpectedly out of shape. If their own public services slowed down in high demand times, it wouldn't look good. The sales channels could try to spin it as “giving you priority over us” but who would genuinely believe them. They could reduce resources available to low priority internal stuff during spikes of other activity, but that would not have a large effect considering the combined size of activity on the public cloud resources.

ramraj07 4y ago

Is it surprising that relatively niche configurations (especially ones which clearly need specific hardware per compute) are finite in aws? But imagine aws running out of m4.X-Large or something. That would be interesting. What scale would you need to run out of the most common instance in a top availability zone? 10k? 100k?