3 ms·

Can I also interpret this as: 'AMD's pytorch support is so abysmal that inference is 10x slower than it should be'?

• croes 3y ago
  Should it not say PyTorch's AMD support?
  • dannyw 3y ago
    It takes two to tango. AMD is always welcome to contribute patches. You also have to keep in mind some latest gen AMD GPUs don’t even officially support ROCm on Linux. That’s absurd. AMD has a choice to invest more staff into ML support, they’re choosing not to.
    • m00x 3y ago
      From Geohotz's investigation in the matter, it doesn't look like it's a manpower issue, it's a quality/culture issue. AMD's firmware GPU division isn't amazing.
    • lhl 3y ago
      I've harped on this on the past, the "official" hardware support list is tiny: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.html#linux-supported-gpus https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h... But, it's worth noting there's different levels of "support." With ROCm 5.6, the 7900XT and 7900XTX RDNA3 cards, while not "officially" supported are gfx1100, which have rocBLAS/MIOpen kernels and work w/o jiggerpokery w/ PyTorch nightly and various HIPified inferencers I've tried like ExLlama.