4 ms·

Can someone explain what a spot market is in this context?

• samstave 9y ago
  Spot, in AWS market, as you likely know, is the ability to bid for regional data center underutilized compute capacity... compute that is not otherwise reserved or dedicated to a particular AWS client account. The risk is that if I outbid you, then I can slurp your instances out from under you, based on if they are needed from the inventory to complete my bid request. In context of Netflix, they purchased tons of reserved instances in AWS - these reserved instances will never enter into AWS spot pool, so are dedicated compute power to Netflix. However, Netflix may not use that reserved instance to its full capacity and so it has downtime that Netflix was paying for. To make efficient use of the tons of money paid to reserve that AWS instance exclusively for Netflix needs, but seeing that it was only needed to stream to clients say, 30% of the day, they built an internal compute bidding market at Netflix, so that other jobs could say “hey netflix, I need to run this encoding job and I need a bunch of boxes, out of all our reserved instances, gimme some that are not currently utilized for my job” So they layered their own internal compute request service to maximize the util of their tons of reserved instances, instead of going to additional AWS instances to complete these other jobs...