3 ms·

Unfortunately AMD totally missed the boat on ML. They need to dramatically increase their investment both in hardware and software. My impression in the past ha

Unfortunately AMD totally missed the boat on ML. They need to dramatically increase their investment both in hardware and software. My impression in the past has also been that they lag in performance on the high end, and their OpenGL driver quality is not as good in terms of features and conformance either. Though I am not totally up to date on their latest stuff.

BearOso 6y ago
The OpenGL quality problems are strictly on the Windows side. The open-source OpenGL drivers are very good.
dragontamer 6y ago
I don't think any company, aside from NVidia, is working on FP16 Tensor cores. Not Intel, not AMD. Google has a systolic processor for tensor units, but that's cloud only. I know ML is where the current money is at, but coprocessors in general seem to be applicable to far more problems than just machine learning and graphics.
- bayindirh 6y ago
  Just food for thought: What if AMD actually acquires Xilinix? Everybody thought it's about CPU-FPGA integration but, they may pull something very, very different.
  - dragontamer 6y ago
    I'm not an expert on Xilinx's stuff, but Xilinx's 18-bit x 27-bit multiplier DSP48 architecture still seems quite awkward to me. https://www.xilinx.com/support/documentation/white_papers/wp486-deep-learning-int8.pdf https://www.xilinx.com/support/documentation/white_papers/wp... True, a systolic processor can be built out of these components, but I doubt it'd be anything as fast as a proper matrix-multiplication unit that NVidia is putting out. And that's after the complications associated with FPGA programming. ---------- Xilinx's FPGAs are a combination of LUTs (4-bit or 6-bit look-up tables), Routing, and "DSP Slices" (prefabricated multipliers). The bulk of your math is still going to be performed in the DSP Slice. Where FPGAs win is that their LUTs and Routing tables allow you to create custom high-speed glue-logic between these DSP slices. But when it comes to something like a regular Convolutional neural net, the routing is very straightforward. ------ I'm not really a deep learning expert. I know that FP16 seems to be pushed by NVidia. This INT8 stuff seems odd to me (there's no way INT8 gets the same dynamic range as FP16), but I'm also not really sure if the dynamic range is needed or useful in deep learning applications.
    - klyrs 6y ago
      I've done some systolic processors on Xilinx hardware. It's really dang hard, on their biggest, fanciest chips, to pump the DSPs at anything close to their advertised clock speeds. Maybe if I was smarter, I could do an asynchronous design that would somehow avoid clocking issues, but I suspect that doing async between mismatched clock regions would effectively halve the clock (and that might be optimistic). The only consolation for me is that I/O over PCI ends up being my ultimate bottleneck. If AMD did a full-on integration with their infinity fabric, it could actually be a game changer. For me, in my weirdo niche.
      - dragontamer 6y ago
        I'm pretty sure I/O over PCIe is a substantial bottleneck in many applications (which is part of the reason for the 300GB/s POWER9 / OpenCAPI / NVLink / GPUs). Its a shame that NVLink is isolated to pretty much the DGX-line of computers and POWER9 (The "big" ones too, I don't think Talos II supports NVLink). But I guess an interconnect that fast will require dedicated, non-standard hardware. ---------- AMD's GPU Infinity Fabric seems to be a different infinity fabric from their CPU. CPU Infinity Fabric is ~50GB/s, but the GPU Infinity Fabric is GPU-to-GPU links of ~90GB/s. Realistically, PCIe 4.0 and 5.0 will be the path forward for standard PCs. There will always be incredible application-specific solutions (such as NVLink / NVSwitch on the DGX), but the general computer user will wait for the open standards to support such speeds before adopting them. At the moment, GPU-to-GPU communications probably will scale better than CPU-to-GPU comms. CPUs just aren't built for the bandwidth (outside of POWER9 / POWER10).
StillBored 6y ago
Unfortunately AMD totally missed the boat on ML. I think by this measure so did everyone else. Nvidia pushed cuda, and everyone took it, even though 8 or so years ago openCL was a reasonable competitor. But various orgs choose to make a proprietary technology a first class citizen rather than putting the effort into an open alternative. Big win for nvidia, big loss for everyone else not large enough to create their own ML frameworks and accelerators.
- dragontamer 6y ago
  Its not really about CUDA or OpenCL. Its about the choice to support FP16 matrix-multiplication at the assembly level of NVidia chips. When you build a processor that literally has "do 4x4 matrix multiplication" as an assembly statement, you'll get far superior ML performance compared to everyone else. (Except Google's TPUs, which have larger matrix-multiplication arrays)

BearOso 6y ago

The OpenGL quality problems are strictly on the Windows side. The open-source OpenGL drivers are very good.

dragontamer 6y ago

I don't think any company, aside from NVidia, is working on FP16 Tensor cores. Not Intel, not AMD. Google has a systolic processor for tensor units, but that's cloud only. I know ML is where the current money is at, but coprocessors in general seem to be applicable to far more problems than just machine learning and graphics.

bayindirh 6y ago
Just food for thought: What if AMD actually acquires Xilinix? Everybody thought it's about CPU-FPGA integration but, they may pull something very, very different.
- dragontamer 6y ago
  I'm not an expert on Xilinx's stuff, but Xilinx's 18-bit x 27-bit multiplier DSP48 architecture still seems quite awkward to me. https://www.xilinx.com/support/documentation/white_papers/wp486-deep-learning-int8.pdf https://www.xilinx.com/support/documentation/white_papers/wp... True, a systolic processor can be built out of these components, but I doubt it'd be anything as fast as a proper matrix-multiplication unit that NVidia is putting out. And that's after the complications associated with FPGA programming. ---------- Xilinx's FPGAs are a combination of LUTs (4-bit or 6-bit look-up tables), Routing, and "DSP Slices" (prefabricated multipliers). The bulk of your math is still going to be performed in the DSP Slice. Where FPGAs win is that their LUTs and Routing tables allow you to create custom high-speed glue-logic between these DSP slices. But when it comes to something like a regular Convolutional neural net, the routing is very straightforward. ------ I'm not really a deep learning expert. I know that FP16 seems to be pushed by NVidia. This INT8 stuff seems odd to me (there's no way INT8 gets the same dynamic range as FP16), but I'm also not really sure if the dynamic range is needed or useful in deep learning applications.
  - klyrs 6y ago
    I've done some systolic processors on Xilinx hardware. It's really dang hard, on their biggest, fanciest chips, to pump the DSPs at anything close to their advertised clock speeds. Maybe if I was smarter, I could do an asynchronous design that would somehow avoid clocking issues, but I suspect that doing async between mismatched clock regions would effectively halve the clock (and that might be optimistic). The only consolation for me is that I/O over PCI ends up being my ultimate bottleneck. If AMD did a full-on integration with their infinity fabric, it could actually be a game changer. For me, in my weirdo niche.
    - dragontamer 6y ago
      I'm pretty sure I/O over PCIe is a substantial bottleneck in many applications (which is part of the reason for the 300GB/s POWER9 / OpenCAPI / NVLink / GPUs). Its a shame that NVLink is isolated to pretty much the DGX-line of computers and POWER9 (The "big" ones too, I don't think Talos II supports NVLink). But I guess an interconnect that fast will require dedicated, non-standard hardware. ---------- AMD's GPU Infinity Fabric seems to be a different infinity fabric from their CPU. CPU Infinity Fabric is ~50GB/s, but the GPU Infinity Fabric is GPU-to-GPU links of ~90GB/s. Realistically, PCIe 4.0 and 5.0 will be the path forward for standard PCs. There will always be incredible application-specific solutions (such as NVLink / NVSwitch on the DGX), but the general computer user will wait for the open standards to support such speeds before adopting them. At the moment, GPU-to-GPU communications probably will scale better than CPU-to-GPU comms. CPUs just aren't built for the bandwidth (outside of POWER9 / POWER10).

StillBored 6y ago

Unfortunately AMD totally missed the boat on ML. I think by this measure so did everyone else. Nvidia pushed cuda, and everyone took it, even though 8 or so years ago openCL was a reasonable competitor. But various orgs choose to make a proprietary technology a first class citizen rather than putting the effort into an open alternative. Big win for nvidia, big loss for everyone else not large enough to create their own ML frameworks and accelerators.

dragontamer 6y ago
Its not really about CUDA or OpenCL. Its about the choice to support FP16 matrix-multiplication at the assembly level of NVidia chips. When you build a processor that literally has "do 4x4 matrix multiplication" as an assembly statement, you'll get far superior ML performance compared to everyone else. (Except Google's TPUs, which have larger matrix-multiplication arrays)