3 ms·

Aren't Amazon, Google, and Meta running their own silicon for some training and inference? Does MS have an equivalent? That could explain a large part of the g

by gundmc 2y ago

Aren't Amazon, Google, and Meta running their own silicon for some training and inference? Does MS have an equivalent? That could explain a large part of the gap. Edit: it looks like Microsoft announced their own last year, but I can imagine they may be behind the curve in capability and scale out compared to the others

wing-_-nuts 2y ago
Yeah the number of tpus google has alone massively dwarfs the compute anybody else has.
ilove196884 2y ago
Meta is still GPUs. Amazon Trainium 1 was a failure and is trying an upgrade with Trainium 2. Google is a TPU shop but still busy GPUs for cloud.
newfocogi 2y ago
My outsider's understanding is Google is the only one whose custom silicon is the primary compute for their flagship foundation models. I didn't see any messaging about the Nova models being trained on Traininium (AWS), and Meta still talks about the number of H110's training their Llama models.
not2b 2y ago
I was writing a comment saying the same thing when your comment appeared. Yest, Meta, Google, and Amazon all have custom silicon, and it seems Microsoft's similar efforts came later. None of these companies want to give Nvidia all of the money, so going forward, I think Nvidia isn't going to see more competition from these efforts. The big players aren't going to sell their chips to others (I don't think), but they'll make them available to cloud customers.
bee_rider 2y ago
Maybe some other company will catch up. But it is hard. Intel is better at chip design than any of those companies. They spent a lot of effort coming up with a very clever chip that competed well against the current generation of Nvidia chips, while still running your old x86 codes. Nvidia continued increasing memory bandwidth, and nobody cared about Knights Whatever.
summerlight 2y ago
Amazon and Meta is at an early stage for their TPU equivalent and I don't think they're ready for production loads. Only Google has comparable silicons but I suspect even Google TPU are mostly for internal products rather than consumers.

by gundmc 2y ago

wing-_-nuts 2y ago

Yeah the number of tpus google has alone massively dwarfs the compute anybody else has.

ilove196884 2y ago

Meta is still GPUs. Amazon Trainium 1 was a failure and is trying an upgrade with Trainium 2. Google is a TPU shop but still busy GPUs for cloud.

newfocogi 2y ago

My outsider's understanding is Google is the only one whose custom silicon is the primary compute for their flagship foundation models. I didn't see any messaging about the Nova models being trained on Traininium (AWS), and Meta still talks about the number of H110's training their Llama models.

not2b 2y ago

I was writing a comment saying the same thing when your comment appeared. Yest, Meta, Google, and Amazon all have custom silicon, and it seems Microsoft's similar efforts came later. None of these companies want to give Nvidia all of the money, so going forward, I think Nvidia isn't going to see more competition from these efforts. The big players aren't going to sell their chips to others (I don't think), but they'll make them available to cloud customers.

bee_rider 2y ago

Maybe some other company will catch up. But it is hard. Intel is better at chip design than any of those companies. They spent a lot of effort coming up with a very clever chip that competed well against the current generation of Nvidia chips, while still running your old x86 codes. Nvidia continued increasing memory bandwidth, and nobody cared about Knights Whatever.

summerlight 2y ago

Amazon and Meta is at an early stage for their TPU equivalent and I don't think they're ready for production loads. Only Google has comparable silicons but I suspect even Google TPU are mostly for internal products rather than consumers.