5 ms·

So what functionality does Ryzen AI have? I couldn't get past the marketing copy while trying to find it.

by im3w1l 3y ago

p_l 3y ago
It's a 4x4 configuration of the separately available Xilinx/AMD AIE-ML accelerator, available on Xilinx alveo platform IIRC. VLIW vector units optimized for AI (-ML suffix) with runtime reconfigurable network on a chip letting you optimize "streaming" the data between individual cores. The way "AMD IPU" device was implemented, which embeds the system in some new AMD CPUs, the previous drivers (which were Linux/RTOS only) didn't work. I was actually in the middle of reverse engineering how it was exposed in Phoenix (7940hs) APUs to write custom driver for Linux based on the one shipped for windows.
- 3abiton 3y ago
  How does it compare to cuda in terms of versatility and compatibility? most AI frameworks are built around cuda (pytorch, etc..) which is why nvidia has a monopoly now, I fail to see AMD breaking through, but one can hope. Do you see a light at the end of the tunnel?
  - p_l 3y ago
    It's targeting the high power efficiency part of the spectrum where nvidia is much less available. There's some reasonably capable software support, in fact the linux driver linked is significantly more capable than the "SDK" they released so far for windows. Which makes sense, because under Xilinx brand they have been selling accelerators and SoCs using AIE and AIE-ML cores for various use cases in embedded world, and XRT lets you program those with pretty much plain C++. For some AI tasks, they have provided high-end wrappers for models that use ONNX (iirc) so those can be used immediately. This is not ROCm, and has essentially none of the ROCm bullshit in terms of support. I seem to recall there's some work to integrate parts of ROCm (opencl, HIP) with XRT. Definitely going to build some kernels with this and test drive it ASAP.
halJordan 3y ago
Ryzen AI is afaik just the new npu. That's all.
trynumber9 3y ago
Simple Google search suggests these results https://ryzenai.docs.amd.com/en/latest/ https://ryzenai.docs.amd.com/en/latest/ https://xilinx.github.io/mlir-aie/ https://xilinx.github.io/mlir-aie/ Looks like it's another NPU/TPU. Apparently it has a similar architecture as Versal.
roenxi 3y ago
[flagged]
- eightysixfour 3y ago
  It is a matrix math accelerator. The 7000 series of CPUs had one that could perform 10TOPS with models quantized to INT8, and the 8000 series can do 16TOPS. ONNX is a portable format for machine learning models. The developer documentation is here: https://ryzenai.docs.amd.com/en/latest/ https://ryzenai.docs.amd.com/en/latest/
  - roenxi 3y ago
    The developer documentation is there. I skimmed through it. It doesn't explain what "Ryzen AI" does; unless you happen to already know what it does. Consider: > Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops powered by Ryzen AI processors using ONNX Runtime. Realise that "[not] Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops [not] powered by Ryzen AI processors using [other] Runtimes.", because pretty much everyone can run this stuff without Ryzen AI, including AMD customers. And then conclude that they haven't explained what Ryzen AI does. I'm sure they've done something (that something being a matrix math accelerator makes a lot of sense, thanks), but they are assuming the reader already has a lot of context. The changelog gets very excited about ONNX which is how I figured that part out.
    - eightysixfour 3y ago
      Is your argument “if I remove the context and then negate the details, then the sentence is meaningless…”? Sure, I agree with that. There is plenty of information in the statement: > Ryzen AI software lets developers take machine learning models […] and deploy them on laptops powered by Ryzen AI processors […]. If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor.

So what functionality does Ryzen AI have? I couldn't get past the marketing copy while trying to find it.

by im3w1l 3y ago

So what functionality does Ryzen AI have? I couldn't get past the marketing copy while trying to find it.

p_l 3y ago

It's a 4x4 configuration of the separately available Xilinx/AMD AIE-ML accelerator, available on Xilinx alveo platform IIRC. VLIW vector units optimized for AI (-ML suffix) with runtime reconfigurable network on a chip letting you optimize "streaming" the data between individual cores. The way "AMD IPU" device was implemented, which embeds the system in some new AMD CPUs, the previous drivers (which were Linux/RTOS only) didn't work. I was actually in the middle of reverse engineering how it was exposed in Phoenix (7940hs) APUs to write custom driver for Linux based on the one shipped for windows.

3abiton 3y ago
How does it compare to cuda in terms of versatility and compatibility? most AI frameworks are built around cuda (pytorch, etc..) which is why nvidia has a monopoly now, I fail to see AMD breaking through, but one can hope. Do you see a light at the end of the tunnel?
- p_l 3y ago
  It's targeting the high power efficiency part of the spectrum where nvidia is much less available. There's some reasonably capable software support, in fact the linux driver linked is significantly more capable than the "SDK" they released so far for windows. Which makes sense, because under Xilinx brand they have been selling accelerators and SoCs using AIE and AIE-ML cores for various use cases in embedded world, and XRT lets you program those with pretty much plain C++. For some AI tasks, they have provided high-end wrappers for models that use ONNX (iirc) so those can be used immediately. This is not ROCm, and has essentially none of the ROCm bullshit in terms of support. I seem to recall there's some work to integrate parts of ROCm (opencl, HIP) with XRT. Definitely going to build some kernels with this and test drive it ASAP.

halJordan 3y ago

Ryzen AI is afaik just the new npu. That's all.

trynumber9 3y ago

Simple Google search suggests these results https://ryzenai.docs.amd.com/en/latest/ https://ryzenai.docs.amd.com/en/latest/ https://xilinx.github.io/mlir-aie/ https://xilinx.github.io/mlir-aie/ Looks like it's another NPU/TPU. Apparently it has a similar architecture as Versal.

roenxi 3y ago

[flagged]

eightysixfour 3y ago
It is a matrix math accelerator. The 7000 series of CPUs had one that could perform 10TOPS with models quantized to INT8, and the 8000 series can do 16TOPS. ONNX is a portable format for machine learning models. The developer documentation is here: https://ryzenai.docs.amd.com/en/latest/ https://ryzenai.docs.amd.com/en/latest/
- roenxi 3y ago
  The developer documentation is there. I skimmed through it. It doesn't explain what "Ryzen AI" does; unless you happen to already know what it does. Consider: > Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops powered by Ryzen AI processors using ONNX Runtime. Realise that "[not] Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops [not] powered by Ryzen AI processors using [other] Runtimes.", because pretty much everyone can run this stuff without Ryzen AI, including AMD customers. And then conclude that they haven't explained what Ryzen AI does. I'm sure they've done something (that something being a matrix math accelerator makes a lot of sense, thanks), but they are assuming the reader already has a lot of context. The changelog gets very excited about ONNX which is how I figured that part out.
  - eightysixfour 3y ago
    Is your argument “if I remove the context and then negate the details, then the sentence is meaningless…”? Sure, I agree with that. There is plenty of information in the statement: > Ryzen AI software lets developers take machine learning models […] and deploy them on laptops powered by Ryzen AI processors […]. If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor.