5 ms·

It seems all cutting edge datacenters like x.ai Colossus are using Nvidia networking. Now Google is upgrading to Nvidia networking, too. Since Nvidia owns most

It seems all cutting edge datacenters like x.ai Colossus are using Nvidia networking. Now Google is upgrading to Nvidia networking, too. Since Nvidia owns most of the Gpgpu products, they have top notch networking and interconnect, I wonder if they don't have a plan to own all datacenter hardware in the future. Maybe they plan to also release CPUs, motherboards, storage and whatever else is needed.

timenova 2y ago
That was their plan with trying to buy ARM...
Kab1r 2y ago
Grace Hopper already includes Arm based CPUs (and reference motherboards)
danpalmer 2y ago
I read this slightly differently, that specific machine types with Nvidia GPU hardware also have Nvidia networking for tying together those GPUs. Google has its own TPUs and don’t really use GPUs except to sell them to end customers on cloud I think. So using Nvidia networking for Nvidia GPUs across many machines on cloud is really just a reflection of what external customers want to buy. Disclaimer, I work at Google but have no non public info about this.
- dmacedo 2y ago
  Having just worked with some of the Thread folks at M&S, thought I'd reach out and say hello. Seems like it was an awesome team! (=
  - danpalmer 2y ago
    You're lucky to be working with them, an amazing team.
adrian_b 2y ago
Nvidia networking is what used to be called Mellanox networking, which was already dominant in datacenters.
- immibis 2y ago
  Only within supercomputers (including the smaller GPU ones used to train AI). Normal data centers use Cisco or Juniper or similarly.well known Ethernet equipment, and they still do. The Mellanox/Nvidia Infiniband networks are specifically used for supercomputer-like clusters.
  - wbl 2y ago
    Mellanox Ethernet NIC got used a bunch of places due to better programmability.
  - dafugg 2y ago
    You seem to have a narrow definition of “normal” for datacenters. Meta were using OCP mellanox NICs for common hardware platforms a decade ago and still are.
  - crmd 2y ago
    Mellanox IB were ubiquitous in storage networking. None of the storage systems I worked on would have been possible without mellanox tech.
  - wil421 2y ago
    Most people on the TrueNAS/FreeNAS forums use 10gb Mellanox nics that are sold on eBay. The sellers get them after server gear gets retired.
HDThoreaun 2y ago
I have to wonder if Nvidia has reached a point where it hesitates to develop new products because it would hurt their margins. Sure they could probably release a profitable networking product but if they did their net margins would decrease even as profit increased. This may actually hurt their market cap as investors absolutely love high margins.
- eru 2y ago
  They can always release capital back to investors, and then those investors can put the money into different companies that eg produce networking equipment.
  - thrw42A8N 2y ago
    Why would they release money if they can invest it and return much more?
    - eru 2y ago
      I was working under HDThoreaun's assumption that the margins would be lower. If they have other opportunities for investment with higher margins, they should seize those, of course. And perhaps even call up investors for more capital, if required.
      - lovich 2y ago
        When the agents employed by investors would be harmed by releasing capital back, which is guaranteed since so many people’s compensation is in the form of stock and returning capital leads to decreasing stock value, why would those agents ever return the capital voluntarily?
        eru 2y ago
        Have you heard of stock buybacks?
        2y ago
mikeyouse 2y ago
Yeah there’s a bit of industry worry about that very eventuality — hence the ultra Ethernet consortium trying to work on open source alternatives to the mellanox/nvidia lock-in. https://ultraethernet.org/ https://ultraethernet.org/
- ravetcofx 2y ago
  Interesting Nvidia is on the steering committee
  - jclulow 2y ago
    Cisco have sat on the steering committees for a lot of things where they had a proprietary initial version of something. It's not that unusual, and also, it's often frankly not actually that open; e.g., see the rent seeking racket for access to PCI documentation, or USB-IF actively seeking to prevent open source hardware from existing, etc.
    - mikeyouse 2y ago
      Eh, the UEC effort is a standards org through the Linux Foundation so it won't be subject to any of the usual chicanery. And actually, it looks like Nvidia is jsut a general member and not one of the Steering Committee members; https://ultraethernet.org/wp-content/uploads/sites/20/2023/09/23.08.10-UEC-Overview-Presentation-FINAL-2.pdf https://ultraethernet.org/wp-content/uploads/sites/20/2023/0...
jonas21 2y ago
I believe this is what they plan on doing. See, for example: https://www.youtube.com/live/Y2F8yisiS6E?si=GbyzzIG8w-mtS7s-&t=1598 https://www.youtube.com/live/Y2F8yisiS6E?si=GbyzzIG8w-mtS7s-...

timenova 2y ago

That was their plan with trying to buy ARM...

Kab1r 2y ago

Grace Hopper already includes Arm based CPUs (and reference motherboards)

danpalmer 2y ago

I read this slightly differently, that specific machine types with Nvidia GPU hardware also have Nvidia networking for tying together those GPUs. Google has its own TPUs and don’t really use GPUs except to sell them to end customers on cloud I think. So using Nvidia networking for Nvidia GPUs across many machines on cloud is really just a reflection of what external customers want to buy. Disclaimer, I work at Google but have no non public info about this.

dmacedo 2y ago
Having just worked with some of the Thread folks at M&S, thought I'd reach out and say hello. Seems like it was an awesome team! (=
- danpalmer 2y ago
  You're lucky to be working with them, an amazing team.

adrian_b 2y ago

Nvidia networking is what used to be called Mellanox networking, which was already dominant in datacenters.

immibis 2y ago
Only within supercomputers (including the smaller GPU ones used to train AI). Normal data centers use Cisco or Juniper or similarly.well known Ethernet equipment, and they still do. The Mellanox/Nvidia Infiniband networks are specifically used for supercomputer-like clusters.
- wbl 2y ago
  Mellanox Ethernet NIC got used a bunch of places due to better programmability.
- dafugg 2y ago
  You seem to have a narrow definition of “normal” for datacenters. Meta were using OCP mellanox NICs for common hardware platforms a decade ago and still are.
- crmd 2y ago
  Mellanox IB were ubiquitous in storage networking. None of the storage systems I worked on would have been possible without mellanox tech.
- wil421 2y ago
  Most people on the TrueNAS/FreeNAS forums use 10gb Mellanox nics that are sold on eBay. The sellers get them after server gear gets retired.

HDThoreaun 2y ago

I have to wonder if Nvidia has reached a point where it hesitates to develop new products because it would hurt their margins. Sure they could probably release a profitable networking product but if they did their net margins would decrease even as profit increased. This may actually hurt their market cap as investors absolutely love high margins.

eru 2y ago
They can always release capital back to investors, and then those investors can put the money into different companies that eg produce networking equipment.
- thrw42A8N 2y ago
  Why would they release money if they can invest it and return much more?
  - eru 2y ago
    I was working under HDThoreaun's assumption that the margins would be lower. If they have other opportunities for investment with higher margins, they should seize those, of course. And perhaps even call up investors for more capital, if required.
    - lovich 2y ago
      When the agents employed by investors would be harmed by releasing capital back, which is guaranteed since so many people’s compensation is in the form of stock and returning capital leads to decreasing stock value, why would those agents ever return the capital voluntarily?
      - eru 2y ago
        Have you heard of stock buybacks?
      - 2y ago

mikeyouse 2y ago

Yeah there’s a bit of industry worry about that very eventuality — hence the ultra Ethernet consortium trying to work on open source alternatives to the mellanox/nvidia lock-in. https://ultraethernet.org/ https://ultraethernet.org/

ravetcofx 2y ago
Interesting Nvidia is on the steering committee
- jclulow 2y ago
  Cisco have sat on the steering committees for a lot of things where they had a proprietary initial version of something. It's not that unusual, and also, it's often frankly not actually that open; e.g., see the rent seeking racket for access to PCI documentation, or USB-IF actively seeking to prevent open source hardware from existing, etc.
  - mikeyouse 2y ago
    Eh, the UEC effort is a standards org through the Linux Foundation so it won't be subject to any of the usual chicanery. And actually, it looks like Nvidia is jsut a general member and not one of the Steering Committee members; https://ultraethernet.org/wp-content/uploads/sites/20/2023/09/23.08.10-UEC-Overview-Presentation-FINAL-2.pdf https://ultraethernet.org/wp-content/uploads/sites/20/2023/0...

jonas21 2y ago

I believe this is what they plan on doing. See, for example: https://www.youtube.com/live/Y2F8yisiS6E?si=GbyzzIG8w-mtS7s-&t=1598 https://www.youtube.com/live/Y2F8yisiS6E?si=GbyzzIG8w-mtS7s-...