5 ms·

I don't have a horse in this race, but I saw today that there was a healthy thread <https://news.ycombinator.com/item?id=40011762 https://news.ycombinator.com/i

I don't have a horse in this race, but I saw today that there was a healthy thread <https://news.ycombinator.com/item?id=40011762 https://news.ycombinator.com/item?id=40011762> about "just pull fiber" when folks were discussing pulling cat6e and a lot of that talk was about the relative cost versus massive bandwidth gains. My mental model of GPU interconnect is that one wishes to have the most bandwidth possible

westurner 3y ago
https://www.windowscentral.com/hardware/computers-desktops/nvidia-kills-off-nvlink-on-rtx-4090 https://www.windowscentral.com/hardware/computers-desktops/n... : > "The reason why we took [NVLink] out was because we needed the we needed the I/Os for something else, and so, so we use the I/O area to cram in as much as much AI processing as we could," Huang confirmed and explained of NVLink's absence. > For gamers, creators, and workstation users who want a multi-GPU setup with the latest RTX 4000 series, Nvidia is switching from NVLink as a bridge to the PCIe Gen 5 standard. PCIe 5.0 x32 supports 128GB/s. "PCIe 7.0 Draft 0.5 Spec: 512 GB/s over PCIe x16 On Track For 2025" https://news.ycombinator.com/item?id=39933959 https://news.ycombinator.com/item?id=39933959 > The HBM3E [RAM] Wikipedia article says 1.2TB/s (2024) https://news.ycombinator.com/item?id=39938652 https://news.ycombinator.com/item?id=39938652 "Fiber-optic data transfer speeds hit a rapid 301 Tbps" (2024-03) https://news.ycombinator.com/item?id=39864107 https://news.ycombinator.com/item?id=39864107 HPE Cray Slingshot is a copper Ethernet HPC interconnect: https://www.nextplatform.com/2022/01/31/crays-slingshot-interconnect-is-at-the-heart-of-hpes-hpc-and-ai-ambitions/?amp https://www.nextplatform.com/2022/01/31/crays-slingshot-inte... ... "HPE Slingshot Makes The GPUs Do Control Plane Compute" https://www.nextplatform.com/2022/08/15/hpe-slingshot-makes-the-gpus-do-control-plane-compute/?amp https://www.nextplatform.com/2022/08/15/hpe-slingshot-makes-... : > Here’s the gist. With normal GPU aware MPI software, inside the node the GPU vendors – the only two that matter today are Nvidia and AMD – have their own mechanisms for doing peer-to-peer communications between the GPUs – in this case NVLink and Infinity Fabric. For MPI data exchanges between GPUs located in different nodes, which is often required for running large simulations and models as well as for large AI training runs, MPI data movement is done through Remote Direct Memory Access methods, pioneered with InfiniBand adapters decades ago, which allow for the transfer of data between a GPU and a network interface card without the interaction of the host CPU’s network stack. [...] > In a [recent paper], HPE showed off the new MPI offload method, making the distinction between the typical MPI communication that is GPU aware versus the stream triggered method that is GPU stream aware. "Exploring GPU Stream-Aware Message Passing using Triggered Operations" (2022) https://arxiv.org/abs/2208.04817 https://arxiv.org/abs/2208.04817 https://scholar.google.com/scholar?cites=13547610567386067445&as_sdt=5,43&sciodt=0,43&hl=en https://scholar.google.com/scholar?cites=1354761056738606744...
- westurner 3y ago
  "New RISC-V microprocessor can run CPU, GPU, and NPU workloads simultaneously" (2024) https://news.ycombinator.com/item?id=39938538 https://news.ycombinator.com/item?id=39938538
- justinclift 3y ago
  > ... with the latest RTX 4000 series, Nvidia is switching from NVLink as a bridge to the PCIe Gen 5 standard. The theory sounds nice, but isn't the RTX 4090 only PCIe gen 4, not 5?
  - gmerc 3y ago
    That’s the beauty of it.
    - justinclift 3y ago
      Not really sure what you're meaning?
      - cyanydeez 3y ago
        Pushing plebs to buy faster gpus requires limiting consumer options
blagie 3y ago
* Wired ethernet is now usually slower than my 802.11. 802.11ax is 0.5-10Gbps in theory, and in practice, faster than gigabit ethernet. * Fiber seems... overpriced. I'm not quite sure why. USB TOSlink is $3 on Aliexpress, as is the cable, so there's nothing fundamentally expensive there, except for being enterprisey. A lot more interconnect would make sense to have as optical. * Wifi is increasing in speed. Wifi 8 is planned to hit 100Gbps. That's where the investment seems to be going.
- rowanG077 3y ago
  How is the latency and consistency on these newer WiFi standards? From my experience if you hit the sweet spot on WiFi it's very good. But then someone else starts a youtube video. Or you move your laptop. Or maybe even do nothing noticeable at all. And suddenly you have a shit experience. For this reason I have always preferred Ethernet. But I admit I have not compared the more recent WiFi standards.
  - blagie 3y ago
    That's actually what's surprised me. In my home, I found wifi to work better in all respects than ethernet. This is a relatively recent phenomenon. Ethernet has issues too, at times. I've certainly seen nominal 10Gbps networks degrade to <1Gbps, and to fix that, you need to debug wires and connectors, and see where the signal integrity issue too. As speeds go up, those issues get more common. 10Mbps, you could almost literally run a pair of wires from Radio Shack and it would work. 1Gbps was the breakpoint where if I didn't use proper everything (which is often mislabelled on Amazon / Aliexpress / eBay / etc.), things would degrade. On the other hand, my wifi has a nice MIMO router. I expect that split will continue. In theory, a MIMO router with an array of antennas could be almost point-to-point, directing signals where they need to go. At the same time, as speeds go up, any parasitic cable / connector inductance or capacitance will have increasing impact.
- vrighter 3y ago
  toslink is a piece of junk plastic fiber and a red led. it's technically fiber optic, but can't really be compared to real fiber optic system

westurner 3y ago

https://www.windowscentral.com/hardware/computers-desktops/nvidia-kills-off-nvlink-on-rtx-4090 https://www.windowscentral.com/hardware/computers-desktops/n... : > "The reason why we took [NVLink] out was because we needed the we needed the I/Os for something else, and so, so we use the I/O area to cram in as much as much AI processing as we could," Huang confirmed and explained of NVLink's absence. > For gamers, creators, and workstation users who want a multi-GPU setup with the latest RTX 4000 series, Nvidia is switching from NVLink as a bridge to the PCIe Gen 5 standard. PCIe 5.0 x32 supports 128GB/s. "PCIe 7.0 Draft 0.5 Spec: 512 GB/s over PCIe x16 On Track For 2025" https://news.ycombinator.com/item?id=39933959 https://news.ycombinator.com/item?id=39933959 > The HBM3E [RAM] Wikipedia article says 1.2TB/s (2024) https://news.ycombinator.com/item?id=39938652 https://news.ycombinator.com/item?id=39938652 "Fiber-optic data transfer speeds hit a rapid 301 Tbps" (2024-03) https://news.ycombinator.com/item?id=39864107 https://news.ycombinator.com/item?id=39864107 HPE Cray Slingshot is a copper Ethernet HPC interconnect: https://www.nextplatform.com/2022/01/31/crays-slingshot-interconnect-is-at-the-heart-of-hpes-hpc-and-ai-ambitions/?amp https://www.nextplatform.com/2022/01/31/crays-slingshot-inte... ... "HPE Slingshot Makes The GPUs Do Control Plane Compute" https://www.nextplatform.com/2022/08/15/hpe-slingshot-makes-the-gpus-do-control-plane-compute/?amp https://www.nextplatform.com/2022/08/15/hpe-slingshot-makes-... : > Here’s the gist. With normal GPU aware MPI software, inside the node the GPU vendors – the only two that matter today are Nvidia and AMD – have their own mechanisms for doing peer-to-peer communications between the GPUs – in this case NVLink and Infinity Fabric. For MPI data exchanges between GPUs located in different nodes, which is often required for running large simulations and models as well as for large AI training runs, MPI data movement is done through Remote Direct Memory Access methods, pioneered with InfiniBand adapters decades ago, which allow for the transfer of data between a GPU and a network interface card without the interaction of the host CPU’s network stack. [...] > In a [recent paper], HPE showed off the new MPI offload method, making the distinction between the typical MPI communication that is GPU aware versus the stream triggered method that is GPU stream aware. "Exploring GPU Stream-Aware Message Passing using Triggered Operations" (2022) https://arxiv.org/abs/2208.04817 https://arxiv.org/abs/2208.04817 https://scholar.google.com/scholar?cites=13547610567386067445&as_sdt=5,43&sciodt=0,43&hl=en https://scholar.google.com/scholar?cites=1354761056738606744...

westurner 3y ago
"New RISC-V microprocessor can run CPU, GPU, and NPU workloads simultaneously" (2024) https://news.ycombinator.com/item?id=39938538 https://news.ycombinator.com/item?id=39938538
justinclift 3y ago
> ... with the latest RTX 4000 series, Nvidia is switching from NVLink as a bridge to the PCIe Gen 5 standard. The theory sounds nice, but isn't the RTX 4090 only PCIe gen 4, not 5?
- gmerc 3y ago
  That’s the beauty of it.
  - justinclift 3y ago
    Not really sure what you're meaning?
    - cyanydeez 3y ago
      Pushing plebs to buy faster gpus requires limiting consumer options

blagie 3y ago

* Wired ethernet is now usually slower than my 802.11. 802.11ax is 0.5-10Gbps in theory, and in practice, faster than gigabit ethernet. * Fiber seems... overpriced. I'm not quite sure why. USB TOSlink is $3 on Aliexpress, as is the cable, so there's nothing fundamentally expensive there, except for being enterprisey. A lot more interconnect would make sense to have as optical. * Wifi is increasing in speed. Wifi 8 is planned to hit 100Gbps. That's where the investment seems to be going.

rowanG077 3y ago
How is the latency and consistency on these newer WiFi standards? From my experience if you hit the sweet spot on WiFi it's very good. But then someone else starts a youtube video. Or you move your laptop. Or maybe even do nothing noticeable at all. And suddenly you have a shit experience. For this reason I have always preferred Ethernet. But I admit I have not compared the more recent WiFi standards.
- blagie 3y ago
  That's actually what's surprised me. In my home, I found wifi to work better in all respects than ethernet. This is a relatively recent phenomenon. Ethernet has issues too, at times. I've certainly seen nominal 10Gbps networks degrade to <1Gbps, and to fix that, you need to debug wires and connectors, and see where the signal integrity issue too. As speeds go up, those issues get more common. 10Mbps, you could almost literally run a pair of wires from Radio Shack and it would work. 1Gbps was the breakpoint where if I didn't use proper everything (which is often mislabelled on Amazon / Aliexpress / eBay / etc.), things would degrade. On the other hand, my wifi has a nice MIMO router. I expect that split will continue. In theory, a MIMO router with an array of antennas could be almost point-to-point, directing signals where they need to go. At the same time, as speeds go up, any parasitic cable / connector inductance or capacitance will have increasing impact.
vrighter 3y ago
toslink is a piece of junk plastic fiber and a red led. it's technically fiber optic, but can't really be compared to real fiber optic system