4 ms·

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even on RDNA1 GPUs) in terms of flexibility. The HIP/CUDA API (nevermind OpenCL) doesn't do them justice, CUDA's enormous ecosystem and vender lock-in notwithstanding.

baybal2 5y ago
That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.
- mkl 5y ago
  More on SLIDE for those curious: https://twimlai.com/slide-the-algorithmic-alternative-that-outperforms-gpus/ https://twimlai.com/slide-the-algorithmic-alternative-that-o... It's a CPU-based deep learning training algorithm that beats GPUs for some tasks.
- freemint 5y ago
  I've not seen anything convincing that slide can approximate convolutional layers like at all without almost doing a full convolution at which point it is not competitive. Can you link me to paper about recent (2021?) work or improvements with slide?
  - fxtentacle 5y ago
    I thought SLIDE was optimized for classification with mind-boggingly sparse data. I.e. "choose the best label for this image out of those 1 mio choices".
  - homerowilson 5y ago
    The current work https://proceedings.mlsys.org/paper/2021/hash/3636638817772e42b59d74cff571fbb3-Abstract.html https://proceedings.mlsys.org/paper/2021/hash/3636638817772e... seems mostly about tuning the original idea instead of expanding its scope. But it's still a neat idea. I guess it could be possible to adpt many of the approximations used in the SLIDE idea to GPUs too though...
- N1H1L 5y ago
  They also have a company called ThirdAI to commercialize SLIDE
cnschn 5y ago
Just curious, what lower level options are there? Inline assembly in OpenCL?
- dragontamer 5y ago
  I never touched it myself, but AMD once exposed the "HSA" interface, which is the AMD GPU execution engine. ROCm / HIP and OpenCL are built on top of that HSA level. I don't think any docs exist for it, but you can see a ton of references to HSA stuff if you browse the ROCm source code. -------- It sounds like the parent post discusses details about how AMD GPUs "pick" the next kernel to run. I've been told that the AMD GPU is very advanced at this, but no adequate interface has ever been exposed to the programmer.
  - diamondlovesyou 5y ago
    Correct; Rocm/HIP/OpenCL are all built on HSA APIs, with a few AMD specific extensions. AMDGPU's command processor is exposed to you (the CP is what invokes "kernels" GPU side): create signals (essentially just an atomic u64 in host memory, with a few extra bells and whistles to support interrupts) and use those in one of the barrier packet types in a HSA device queue. With these (with one sad caveat that work packets can't have deps themselves :( ) you can enqueue most computation graphs and the CP will handle waiting for signals without any CPU involvement. Plus, GPU kernels can also concurrently write to this queue (though you can't create signals GPU side...) AMDGPUs are like shared memory machines, which I think is really cool.
    - dragontamer 5y ago
      > you can enqueue most computation graphs Can those command graphs loop? I know this interface is undocumented... but I've had an idea for a GPU-language akin to Java or Lisp memory-management. The gist is that kernel_X() can execute, but may fail in any new() or malloc() command. In such a case, I'd want the compute-graph to loop: while (kernel_X fails due to out-of-memory){ garbage_collect(); try kernel_X() again}. ------- Not that I have the time to experiment with something like this, but I guess I've been curious to know if that sort of thing can even work.

baybal2 5y ago

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

mkl 5y ago
More on SLIDE for those curious: https://twimlai.com/slide-the-algorithmic-alternative-that-outperforms-gpus/ https://twimlai.com/slide-the-algorithmic-alternative-that-o... It's a CPU-based deep learning training algorithm that beats GPUs for some tasks.
freemint 5y ago
I've not seen anything convincing that slide can approximate convolutional layers like at all without almost doing a full convolution at which point it is not competitive. Can you link me to paper about recent (2021?) work or improvements with slide?
- fxtentacle 5y ago
  I thought SLIDE was optimized for classification with mind-boggingly sparse data. I.e. "choose the best label for this image out of those 1 mio choices".
- homerowilson 5y ago
  The current work https://proceedings.mlsys.org/paper/2021/hash/3636638817772e42b59d74cff571fbb3-Abstract.html https://proceedings.mlsys.org/paper/2021/hash/3636638817772e... seems mostly about tuning the original idea instead of expanding its scope. But it's still a neat idea. I guess it could be possible to adpt many of the approximations used in the SLIDE idea to GPUs too though...
N1H1L 5y ago
They also have a company called ThirdAI to commercialize SLIDE

cnschn 5y ago

Just curious, what lower level options are there? Inline assembly in OpenCL?

dragontamer 5y ago
I never touched it myself, but AMD once exposed the "HSA" interface, which is the AMD GPU execution engine. ROCm / HIP and OpenCL are built on top of that HSA level. I don't think any docs exist for it, but you can see a ton of references to HSA stuff if you browse the ROCm source code. -------- It sounds like the parent post discusses details about how AMD GPUs "pick" the next kernel to run. I've been told that the AMD GPU is very advanced at this, but no adequate interface has ever been exposed to the programmer.
- diamondlovesyou 5y ago
  Correct; Rocm/HIP/OpenCL are all built on HSA APIs, with a few AMD specific extensions. AMDGPU's command processor is exposed to you (the CP is what invokes "kernels" GPU side): create signals (essentially just an atomic u64 in host memory, with a few extra bells and whistles to support interrupts) and use those in one of the barrier packet types in a HSA device queue. With these (with one sad caveat that work packets can't have deps themselves :( ) you can enqueue most computation graphs and the CP will handle waiting for signals without any CPU involvement. Plus, GPU kernels can also concurrently write to this queue (though you can't create signals GPU side...) AMDGPUs are like shared memory machines, which I think is really cool.
  - dragontamer 5y ago
    > you can enqueue most computation graphs Can those command graphs loop? I know this interface is undocumented... but I've had an idea for a GPU-language akin to Java or Lisp memory-management. The gist is that kernel_X() can execute, but may fail in any new() or malloc() command. In such a case, I'd want the compute-graph to loop: while (kernel_X fails due to out-of-memory){ garbage_collect(); try kernel_X() again}. ------- Not that I have the time to experiment with something like this, but I guess I've been curious to know if that sort of thing can even work.