4 ms·

How do Amazon (or any other cloud provider) make sure they have enough of these machines to cope with the demand for them without getting too many?

tachion 10y ago
You pretty much answered your own question. First, they have 'too many', second, if they dont, they oversubscribe, third, after running your business for a while, you can do some math on your stats and predict future demand with accuracy that's 'good enough'.
- raverbashing 10y ago
  I don't think you can oversubscribe GPUs But at the price they're charging, if demand exists, they just call nVidia and order another batch of cards
- DonHopkins 10y ago
  And fortunately, they have enough computing power to do the math.
dmichulke 10y ago
They lowball with the offer (#instances) and have price regulate the demand side. After a few months they can estimate the demand and increase supply accordingly (iff profitable)
dx034 10y ago
A lot of people will use spot instances to get cheaper computer power for non-critical calculations. So they can control demand by the price for spot instances. Also, they'll get a lot of instances up and running before publishing the post. At the moment, p2 spot prices seem very low (~9c for p2.xlarge). Especially in the GPU space, a lot of calculations will likely not be very time-sensitive. The spot market is optimal to control demand and ensure high utilization.
idunno246 10y ago
They estimate, sometimes well sometimes not. Recently Ive seen a region that ran out of an instance type, launching gives you an error, and you're sol for days if you can't adjust. Reserved instances are more than just a price reduction
- iraklism 10y ago
  This is what I'm currently getting in Ireland: Launch Failed We currently do not have sufficient p2.16xlarge capacity in zones with support for 'gp2' volumes. Our system will be working on provisioning additional capacity.
p0rkbelly 10y ago
Capacity planning is a pretty advanced mathematical science, I'm sure there are many Ph.Ds, who are experts in their field, who do this job across the industry.

tachion 10y ago

You pretty much answered your own question. First, they have 'too many', second, if they dont, they oversubscribe, third, after running your business for a while, you can do some math on your stats and predict future demand with accuracy that's 'good enough'.

raverbashing 10y ago
I don't think you can oversubscribe GPUs But at the price they're charging, if demand exists, they just call nVidia and order another batch of cards
DonHopkins 10y ago
And fortunately, they have enough computing power to do the math.

dmichulke 10y ago

They lowball with the offer (#instances) and have price regulate the demand side. After a few months they can estimate the demand and increase supply accordingly (iff profitable)

dx034 10y ago

A lot of people will use spot instances to get cheaper computer power for non-critical calculations. So they can control demand by the price for spot instances. Also, they'll get a lot of instances up and running before publishing the post. At the moment, p2 spot prices seem very low (~9c for p2.xlarge). Especially in the GPU space, a lot of calculations will likely not be very time-sensitive. The spot market is optimal to control demand and ensure high utilization.

idunno246 10y ago

They estimate, sometimes well sometimes not. Recently Ive seen a region that ran out of an instance type, launching gives you an error, and you're sol for days if you can't adjust. Reserved instances are more than just a price reduction

iraklism 10y ago
This is what I'm currently getting in Ireland: Launch Failed We currently do not have sufficient p2.16xlarge capacity in zones with support for 'gp2' volumes. Our system will be working on provisioning additional capacity.

p0rkbelly 10y ago

Capacity planning is a pretty advanced mathematical science, I'm sure there are many Ph.Ds, who are experts in their field, who do this job across the industry.