5 ms·

Unfortunately, the software tooling gap between Nvidia and AMD has widened significantly for GPGPUs. AMD has also left the enthusiast video card market in the d

Unfortunately, the software tooling gap between Nvidia and AMD has widened significantly for GPGPUs. AMD has also left the enthusiast video card market in the dust with their lack of ROCM support. Nvidia has some seriously cool tech that they showcased at GTC 2021 this year, like speeding up algorithms for bioinformatics, and new CPU/GPU combinations for autonomous driving. I would guess that the immense amount of R&D spent by Nvidia is causing this gap.

kd913 5y ago
This is the kind of thing where the whole Xilinx acquisition is gonna make a play. Bespoke accelerator FPGA solutions are the future and will probably scale a lot longer than CPU/GPU combinations.
- davrosthedalek 5y ago
  I really doubt that. FPGA is really power and silicon intensive. A more traditional "many 'fixed function' processor" a la GPU will beat it for any algorithm which at least semi-successfully maps to the provided operations. And most math is matrix multiplication anyway, if you squint your eyes and hold your tongue at the right angle ;) (I use "fixed function" loosely here. It's not a fixed function T&L pipleline, it's meant in the "the circuit is fixed but can execute arbitrary code" sense.)
  - kd913 5y ago
    You can doubt it all you want, but this is where Xilinx Alveo is coming into play. This isn't my space as I operate on low-latency networking side. > A more traditional "many 'fixed function' processor" a la GPU will beat it for any algorithm which at least semi-successfully maps to the provided operations. And most math is matrix multiplication anyway, if you squint your eyes and hold your tongue at the right angle ;) I don't think it is as simple as this. Certainly not for low-latency scenarios or for HPC efficiency where the gains are done my moving as much computation as possible onto accelerator cards and off the CPU. The cost of moving things back and forth from the CPU can be expensive (data especially). We are already seeing dedicated solutions from Google, Graphcore that beats the pants off the traditional GPU/CPU approach.
    - davrosthedalek 5y ago
      How does FPGA fabric solve the bw bottleneck exactly? What solves it is putting the compute power closer to the CPU (or memory). But you can do that with an FPGA or a GPU, or a CPU matrix etc. Compared to FPGA, the others will have much more compute power(especially for HPC) in the same die size/heat envelope.
      - kd913 5y ago
        I think the idea is to bring the compute power closer to the storage or NIC devices. At least that is the impression I am getting from their smartNIC/smartssd device. In this case being video streaming in from the NIC, encoding/decoding occurring directly on the NIC. Same to a degree with the bespoke smartssd where if you have some complex encryption, encoding, decoding or data driven application than the processing happens directly on the storage device via the FPGA.
floatboth 5y ago
It's really disappointing that the GPGPU space isn't adopting Vulkan all that fast. These "compute specific" things such as OpenCL/CUDA/ROCm should not be necessary and should die. Just use Vulkan for fuck's sake.
- rubatuga 5y ago
  Libraries like cublas and cudnn help the GPU scheduler assign tasks to specialized blocks on the graphics card. Vulkan would slow this innovation down severely. Unless they make Vulkan extensions? I’m not too sure how that would work. Personally I haven’t used CUDA, but if anybody could comment on its ease of use compared to Vulkan that would be great.
  - floatboth 5y ago
    > help the GPU scheduler assign tasks to specialized blocks Source? I can't find anything for '"cublas" scheduler'…
    - rubatuga 5y ago
      The GEMM instructions are sent to Tensor Cores by the scheduler. The cutlass library is the open source version of cublas: https://github.com/NVIDIA/cutlass https://github.com/NVIDIA/cutlass
      - floatboth 5y ago
        Seems like you can leverage those via VK_NV_cooperative_matrix…
merb 5y ago
it will get worse, since nvidia lifted some IOMMU limitations in their consumer cards: https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/geforce-gpu-passthrough-for-windows-virtual-machine-%28beta%29 https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/g... this is a big deal especially since ai tech advanced.

kd913 5y ago

This is the kind of thing where the whole Xilinx acquisition is gonna make a play. Bespoke accelerator FPGA solutions are the future and will probably scale a lot longer than CPU/GPU combinations.

davrosthedalek 5y ago
I really doubt that. FPGA is really power and silicon intensive. A more traditional "many 'fixed function' processor" a la GPU will beat it for any algorithm which at least semi-successfully maps to the provided operations. And most math is matrix multiplication anyway, if you squint your eyes and hold your tongue at the right angle ;) (I use "fixed function" loosely here. It's not a fixed function T&L pipleline, it's meant in the "the circuit is fixed but can execute arbitrary code" sense.)
- kd913 5y ago
  You can doubt it all you want, but this is where Xilinx Alveo is coming into play. This isn't my space as I operate on low-latency networking side. > A more traditional "many 'fixed function' processor" a la GPU will beat it for any algorithm which at least semi-successfully maps to the provided operations. And most math is matrix multiplication anyway, if you squint your eyes and hold your tongue at the right angle ;) I don't think it is as simple as this. Certainly not for low-latency scenarios or for HPC efficiency where the gains are done my moving as much computation as possible onto accelerator cards and off the CPU. The cost of moving things back and forth from the CPU can be expensive (data especially). We are already seeing dedicated solutions from Google, Graphcore that beats the pants off the traditional GPU/CPU approach.
  - davrosthedalek 5y ago
    How does FPGA fabric solve the bw bottleneck exactly? What solves it is putting the compute power closer to the CPU (or memory). But you can do that with an FPGA or a GPU, or a CPU matrix etc. Compared to FPGA, the others will have much more compute power(especially for HPC) in the same die size/heat envelope.
    - kd913 5y ago
      I think the idea is to bring the compute power closer to the storage or NIC devices. At least that is the impression I am getting from their smartNIC/smartssd device. In this case being video streaming in from the NIC, encoding/decoding occurring directly on the NIC. Same to a degree with the bespoke smartssd where if you have some complex encryption, encoding, decoding or data driven application than the processing happens directly on the storage device via the FPGA.

floatboth 5y ago

It's really disappointing that the GPGPU space isn't adopting Vulkan all that fast. These "compute specific" things such as OpenCL/CUDA/ROCm should not be necessary and should die. Just use Vulkan for fuck's sake.

rubatuga 5y ago
Libraries like cublas and cudnn help the GPU scheduler assign tasks to specialized blocks on the graphics card. Vulkan would slow this innovation down severely. Unless they make Vulkan extensions? I’m not too sure how that would work. Personally I haven’t used CUDA, but if anybody could comment on its ease of use compared to Vulkan that would be great.
- floatboth 5y ago
  > help the GPU scheduler assign tasks to specialized blocks Source? I can't find anything for '"cublas" scheduler'…
  - rubatuga 5y ago
    The GEMM instructions are sent to Tensor Cores by the scheduler. The cutlass library is the open source version of cublas: https://github.com/NVIDIA/cutlass https://github.com/NVIDIA/cutlass
    - floatboth 5y ago
      Seems like you can leverage those via VK_NV_cooperative_matrix…

merb 5y ago

it will get worse, since nvidia lifted some IOMMU limitations in their consumer cards: https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/geforce-gpu-passthrough-for-windows-virtual-machine-%28beta%29 https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/g... this is a big deal especially since ai tech advanced.