3 ms·

Or one of the cloud providers who doesn't want to pay lock-in prices when they'd rather pay commodity prices

topspin 1y ago
How feasible is this for a cloud operation(s)? I imagine this work requires close collaboration with the architects and proprietary knowledge about the design.
- ants_everywhere 1y ago
  it seems feasible, it's more a matter of how much of a priority it is. I follow Google most closely. They design and manufacture their own accelerators. AWS I know manufactures its own CPUs, but I don't know if they're working on or already have an AI accelerator. Several of the big players are working on OpenXLA, which is designed to abstract and commoditize the GPU layer: https://openxla.org/xla https://openxla.org/xla OpenXLA mentions: > Alibaba, Amazon Web Services, AMD, Apple, Arm, Google, Intel, Meta, and NVIDIA
  - mdaniel 1y ago
    > AWS I know manufactures its own CPUs, but I don't know if they're working on or already have an AI accelerator I believe those are the Inferentia: https://aws.amazon.com/ai/machine-learning/inferentia/ https://aws.amazon.com/ai/machine-learning/inferentia/ > AWS Inferentia chips are designed by AWS to deliver high performance at the lowest cost in Amazon EC2 for your deep learning (DL) and generative AI inference applications but I don't know this second if they're supported by the major frameworks, or what I also didn't recall about https://aws.amazon.com/ai/machine-learning/trainium/ https://aws.amazon.com/ai/machine-learning/trainium/ until I was looking up that page, so it seems they're trying to have a competitor to the TPUs just naming them dumb, because AWS > AWS Trainium chips are a family of AI chips purpose built by AWS for AI training and inference to deliver high performance while reducing costs.
    - ants_everywhere 1y ago
      thanks this is useful! > have a competitor to the TPUs just naming them dumb, because AWS I kind of like "trainium" although "inferentia" I could take or leave. At least it's nice that the names tell you the intended use case.
pjmlp 1y ago
With what software though?
Twirrim 1y ago
Not sure cloud providers will care, all the costs get passed onto the customers. There's already far more demand for GPUs than can be met by the supply chain, too. If they were sitting on excess stock, or struggling to sell, sure.
coredog64 1y ago
The cloud providers all have their own Nvidia alternatives. Having worked with more than one, I would rate them not much better than AMD when it comes to software.

How feasible is this for a cloud operation(s)? I imagine this work requires close collaboration with the architects and proprietary knowledge about the design.

ants_everywhere 1y ago
it seems feasible, it's more a matter of how much of a priority it is. I follow Google most closely. They design and manufacture their own accelerators. AWS I know manufactures its own CPUs, but I don't know if they're working on or already have an AI accelerator. Several of the big players are working on OpenXLA, which is designed to abstract and commoditize the GPU layer: https://openxla.org/xla https://openxla.org/xla OpenXLA mentions: > Alibaba, Amazon Web Services, AMD, Apple, Arm, Google, Intel, Meta, and NVIDIA
- mdaniel 1y ago
  > AWS I know manufactures its own CPUs, but I don't know if they're working on or already have an AI accelerator I believe those are the Inferentia: https://aws.amazon.com/ai/machine-learning/inferentia/ https://aws.amazon.com/ai/machine-learning/inferentia/ > AWS Inferentia chips are designed by AWS to deliver high performance at the lowest cost in Amazon EC2 for your deep learning (DL) and generative AI inference applications but I don't know this second if they're supported by the major frameworks, or what I also didn't recall about https://aws.amazon.com/ai/machine-learning/trainium/ https://aws.amazon.com/ai/machine-learning/trainium/ until I was looking up that page, so it seems they're trying to have a competitor to the TPUs just naming them dumb, because AWS > AWS Trainium chips are a family of AI chips purpose built by AWS for AI training and inference to deliver high performance while reducing costs.
  - ants_everywhere 1y ago
    thanks this is useful! > have a competitor to the TPUs just naming them dumb, because AWS I kind of like "trainium" although "inferentia" I could take or leave. At least it's nice that the names tell you the intended use case.

pjmlp 1y ago

With what software though?

Twirrim 1y ago

Not sure cloud providers will care, all the costs get passed onto the customers. There's already far more demand for GPUs than can be met by the supply chain, too. If they were sitting on excess stock, or struggling to sell, sure.

coredog64 1y ago

The cloud providers all have their own Nvidia alternatives. Having worked with more than one, I would rate them not much better than AMD when it comes to software.