3 ms·
In financial markets, this quirk of automatically-determined prices across different products is frequently called "mispricing" when those products logically sh
by pclmulqdq 4y ago
In financial markets, this quirk of automatically-determined prices across different products is frequently called "mispricing" when those products logically should have a relationship with each other.
Straightforwardly: All hosts with space for a c6gd spot instance have space for a c6g instance. If Amazon is willing to host a c6gd instance in that slot for $X, they should be willing to also host a c6g instance there for $X.
In financial markets, the way this gets handled is through arbitrage: someone will buy the equivalent of the c6gd instance, and sell the c6g part for the higher price (they may also sell the "d" part for even more money). This has the effect of "correcting" the price. The AWS spot market does not allow you to do arbitrage, and AWS doesn't appear to do the arbitrage for you.
AWS probably likes this inefficiency in their market: some instance types are more popular than others, and some customers make assumptions that require them to use a very specific instance type (ie a c6gd would not work as a substitute for their c6g instance). However, the vast majority of users probably could work just fine if their c6g instance were a c6gd, and don't look for the arbitrage opportunity. That means Amazon gets paid extra.
- Moissanite 4y ago> If Amazon is willing to host a c6gd instance in that slot for $X, they should be willing to also host a c6g instance there for $X. The reality is that direct c6gd demand might be an order of magnitude lower than c6g direct demand - if AWS can get some more flexible people to adopt c6gd by offering a lower price, c6g capacity is slightly stabilized for on-demand usage by people who don't value the flexibility. Also note that c6g to c6gd has a non-zero switching cost - extra NVMe on the instance adds a new source of potential hardware failure, increasing the probability of termination very slightly. There might be other software-related costs depending on whether your application makes any ill-advised assumptions about attached storage during setup. So overall, I would just be happier to read this article if it was framed as "PSA: having more features in an ec2 instance is sometimes cheaper! Don't rule yourself out of extra savings by making overly-constrained fleet requests." The extra commentary about foregone revenue makes too many assumptions and detracts from the core point.
- pclmulqdq 4y agoThe point is that Amazon doesn't have to fill that slot with a c6gd. They can also fill it with a c6g. They just choose not to. The fact that you have to host a c6gd to get that price instead of a c6g is an inefficiency in the spot market that likely makes Amazon money, but is a little customer-hostile. I think the article is probably wrong that Amazon is foregoing revenue due to this. This is a form of price discrimination and it is likely making Amazon money, but in a scummy way.
- ericpauley 4y agoAgreed that it's definitely difficult to know the true missed revenue here without internal data, and even then you'd be making some assumptions. I am confident there is some missed revenue here, as amazon routinely has spot capacity constraints under existing prices so could definitely sell some substitute instances without moving the original instance market (even one instance per pool substituted equates to >$1M per year). In either case, a savvy organization can definitely benefit from the price discrepancy even if Amazon couldn't.
- Moissanite 4y agoI can agree that there is missed revenue - but realistically it wouldale much more sense to sell that capacity via Fargate (which is closer to undifferentiated generic compute and RAM) rather than monkeying with the spot pricing algorithm.
- ericpauley 4y agoGreat point on Fargate, I'd be very curious on whether they select capacity for that from EC2 capcity or if there's a separate physical footprint for it.