5 ms·

AWS is so anti-customer with respect to GPUs right now. They have the highest prices of any cloud. What happened to “your margin is my opportunity”? And, as f

by rytill 2y ago

AWS is so anti-customer with respect to GPUs right now. They have the highest prices of any cloud. What happened to “your margin is my opportunity”? And, as far as I know, customers are unable to allocate a VM with fewer than eight A100, H100, or H200 GPUs. (Please tell me how if I’m wrong.) So, customers are incentivized to use other cloud products for GPUs in the short term. They seem to be heavily invested in their own chips in the medium term.

hyperliner 2y ago
[dead]
re-thc 2y ago
They're trying to sell you their managed product, i.e. bedrock.
lpasselin 2y ago
Meanwhile, I had a hard time last week getting a machine with 8 gpus from azure.
- alephnerd 2y ago
  Every customer of a 3rd party cloud provider is going to have a hard time getting Nvidia GPU compute from 3rd party cloud providers - not only is the amount of GPUs available limited, but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor.
  - mattferderer 2y ago
    > but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor I thought Jensen recently said he does not want to offer their own cloud offering. He instead wants to focus on creating ready made solutions for cloud vendors to purchase & re-sell services with.
    - alephnerd 2y ago
      Fair point! I think that's a recentish pivot though (past 2-3 years). I vaguely remember that in the late 2010s they were building and testing DGX Cloud as it's own standalone offering, but I might be wrong and confusing some other offerings they were working on.
electronbeam 2y ago
They typically come physically in 8s as an SXM5 tray. It takes more effort to virtualize SXM5 than individual PCIe cards, so you find availability in 8s in VMs too regardless of provider
JCM9 2y ago
They seem to be sold out of an extremely popular product and seem to be making good money doing so (per AWS’s results). How’s that “anti-customer?” Also, in case you didn’t get the memo GPUs are hard to come by most places these days.

by rytill 2y ago

hyperliner 2y ago

[dead]

re-thc 2y ago

They're trying to sell you their managed product, i.e. bedrock.

lpasselin 2y ago

Meanwhile, I had a hard time last week getting a machine with 8 gpus from azure.

alephnerd 2y ago
Every customer of a 3rd party cloud provider is going to have a hard time getting Nvidia GPU compute from 3rd party cloud providers - not only is the amount of GPUs available limited, but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor.
- mattferderer 2y ago
  > but Nvidia themselves are trying to spin up their own cloud provider offering, and unsurprisingly don't want to help a competitor I thought Jensen recently said he does not want to offer their own cloud offering. He instead wants to focus on creating ready made solutions for cloud vendors to purchase & re-sell services with.
  - alephnerd 2y ago
    Fair point! I think that's a recentish pivot though (past 2-3 years). I vaguely remember that in the late 2010s they were building and testing DGX Cloud as it's own standalone offering, but I might be wrong and confusing some other offerings they were working on.

electronbeam 2y ago

They typically come physically in 8s as an SXM5 tray. It takes more effort to virtualize SXM5 than individual PCIe cards, so you find availability in 8s in VMs too regardless of provider

JCM9 2y ago

They seem to be sold out of an extremely popular product and seem to be making good money doing so (per AWS’s results). How’s that “anti-customer?” Also, in case you didn’t get the memo GPUs are hard to come by most places these days.