3 ms·

It is a matrix math accelerator. The 7000 series of CPUs had one that could perform 10TOPS with models quantized to INT8, and the 8000 series can do 16TOPS. ON

It is a matrix math accelerator. The 7000 series of CPUs had one that could perform 10TOPS with models quantized to INT8, and the 8000 series can do 16TOPS. ONNX is a portable format for machine learning models. The developer documentation is here: https://ryzenai.docs.amd.com/en/latest/ https://ryzenai.docs.amd.com/en/latest/

roenxi 3y ago
The developer documentation is there. I skimmed through it. It doesn't explain what "Ryzen AI" does; unless you happen to already know what it does. Consider: > Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops powered by Ryzen AI processors using ONNX Runtime. Realise that "[not] Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops [not] powered by Ryzen AI processors using [other] Runtimes.", because pretty much everyone can run this stuff without Ryzen AI, including AMD customers. And then conclude that they haven't explained what Ryzen AI does. I'm sure they've done something (that something being a matrix math accelerator makes a lot of sense, thanks), but they are assuming the reader already has a lot of context. The changelog gets very excited about ONNX which is how I figured that part out.
- eightysixfour 3y ago
  Is your argument “if I remove the context and then negate the details, then the sentence is meaningless…”? Sure, I agree with that. There is plenty of information in the statement: > Ryzen AI software lets developers take machine learning models […] and deploy them on laptops powered by Ryzen AI processors […]. If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor.
  - roenxi 3y ago
    The sentence is meaningful both ways. The argument is if [X] and [not X] are both true, X doesn't rule out any possible universes. And if it doesn't rule out any possible universes, it doesn't tell us anything about what they have done - because universes where they didn't do anything are still valid. That is a handy test of whether a sentence is marketing fluff or whether it contains data. > If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor. I can already deploy models to my processor though, and it is an AMD processor that doesn't support Ryzen AI. I've used it to develop AI models too, since I have an AMD GPU and can't use that to develop AI models because it crashes the driver when I do. So while you can deploy models to a Ryzen AI processor, you can also deploy models to non-Ryzen AI processors. So, logically and practically, they haven't told us anything. The issue is with the limited understanding you've taken on for your argument, you didn't realise that all computers can already do what they promised Ryzen AI processors can do. Compare that to eightysixfour who articulates what the thing actually does - lots of TOPS, and a 150% improvement in TOPS on the previous generation. His sentence contains information and I doubt my CPU can match that TOPS performance. And I wasn't developing an INT8 model which is probably also relevant. Furthermore, it is that in 10 years or so models will be much bigger and more power hungry, possibly with different architectures. So there will be models that we won't be able to deploy on many CPUs even if they are powered by Ryzen AI. So not only is the documentation on Ryzen AI not telling me what they can do, even what the documentation promises can't be logically done. Which isn't a problem because that is more of a silly "gotcha!" style observation, but it really highlights how little useful information is in the docs intro material and marketing copy when it comes to identifying what it is this thing does.
    - eightysixfour 3y ago
      First, I am eightysixfour. Second, if someone says “AI processor” the context is quite obvious that it is a dedicated AI accelerator, like a Coral TPU. Especially since the first sentence on the page defines it. > AMD Ryzen™ AI Software enables developers to take full advantage of AMD XDNA™ architecture integrated in select AMD Ryzen AI processors. Third, > you didn't realise that all computers can already do what they promised Ryzen AI processors can do In this text they didn’t promise Ryzen AI processors could do anything, they promised the Ryzen AI software would let you deploy models to Ryzen AI processors. > The issue is with the limited understanding you've taken on for your argument, you didn't realise that all computers can already do what they promised Ryzen AI processors can do. I have deployed machine learning models to the edge in production on accelerators and CPUs. I use Llamas on my home rig, both CPU and CUDA. I am quite aware of what they can do, which is why I already knew the performance of the Ryzen AI processor. You are the one whose limited understanding kept you from recognizing that “Ryzen AI processor” is a distinct thing from a CPU or GPU. Your complaint is the equivalent of complaining that the home page for CUDA’s documentation doesn’t explain what a GPU is and why it is different from a CPU, which can do the same computations. You also seem moderately confused about the use cases for these relatively low performance, but very low power ML accelerators. The first thing that shipped on Ryzen’s AI accelerator is a model from Microsoft that makes it look like you are making eye contact with a camera. Not a giant LLM. They don’t compete with GPUs.
daemonologist 3y ago
For comparison, Google's Edge TPU (found in the Coral USB accelerator for example) will do 4 INT8 TOPS [0], an Nvidia T4 will do 130 [1], and an A100 or A6000 will do 620 [2]. Fully utilized it could be expected to be radically faster and more efficient than CPU but still of course much slower than workstation/server hardware for these operations. [0] https://coral.ai/products/accelerator https://coral.ai/products/accelerator [1] https://semiengineering.com/tops-memory-throughput-and-inference-efficiency/ https://semiengineering.com/tops-memory-throughput-and-infer... [2] https://www.microway.com/knowledge-center-articles/in-depth-comparison-of-nvidia-ampere-gpu-accelerators/ https://www.microway.com/knowledge-center-articles/in-depth-...

roenxi 3y ago

The developer documentation is there. I skimmed through it. It doesn't explain what "Ryzen AI" does; unless you happen to already know what it does. Consider: > Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops powered by Ryzen AI processors using ONNX Runtime. Realise that "[not] Ryzen AI software lets developers take machine learning models trained in PyTorch or TensorFlow and deploy them on laptops [not] powered by Ryzen AI processors using [other] Runtimes.", because pretty much everyone can run this stuff without Ryzen AI, including AMD customers. And then conclude that they haven't explained what Ryzen AI does. I'm sure they've done something (that something being a matrix math accelerator makes a lot of sense, thanks), but they are assuming the reader already has a lot of context. The changelog gets very excited about ONNX which is how I figured that part out.

eightysixfour 3y ago
Is your argument “if I remove the context and then negate the details, then the sentence is meaningless…”? Sure, I agree with that. There is plenty of information in the statement: > Ryzen AI software lets developers take machine learning models […] and deploy them on laptops powered by Ryzen AI processors […]. If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor.
- roenxi 3y ago
  The sentence is meaningful both ways. The argument is if [X] and [not X] are both true, X doesn't rule out any possible universes. And if it doesn't rule out any possible universes, it doesn't tell us anything about what they have done - because universes where they didn't do anything are still valid. That is a handy test of whether a sentence is marketing fluff or whether it contains data. > If I have a passing understanding of computers, I know everything I need to decide if this is interesting to me or not. It is for machine learning developers. It helps them deploy their models to a specific AI processor. I can already deploy models to my processor though, and it is an AMD processor that doesn't support Ryzen AI. I've used it to develop AI models too, since I have an AMD GPU and can't use that to develop AI models because it crashes the driver when I do. So while you can deploy models to a Ryzen AI processor, you can also deploy models to non-Ryzen AI processors. So, logically and practically, they haven't told us anything. The issue is with the limited understanding you've taken on for your argument, you didn't realise that all computers can already do what they promised Ryzen AI processors can do. Compare that to eightysixfour who articulates what the thing actually does - lots of TOPS, and a 150% improvement in TOPS on the previous generation. His sentence contains information and I doubt my CPU can match that TOPS performance. And I wasn't developing an INT8 model which is probably also relevant. Furthermore, it is that in 10 years or so models will be much bigger and more power hungry, possibly with different architectures. So there will be models that we won't be able to deploy on many CPUs even if they are powered by Ryzen AI. So not only is the documentation on Ryzen AI not telling me what they can do, even what the documentation promises can't be logically done. Which isn't a problem because that is more of a silly "gotcha!" style observation, but it really highlights how little useful information is in the docs intro material and marketing copy when it comes to identifying what it is this thing does.
  - eightysixfour 3y ago
    First, I am eightysixfour. Second, if someone says “AI processor” the context is quite obvious that it is a dedicated AI accelerator, like a Coral TPU. Especially since the first sentence on the page defines it. > AMD Ryzen™ AI Software enables developers to take full advantage of AMD XDNA™ architecture integrated in select AMD Ryzen AI processors. Third, > you didn't realise that all computers can already do what they promised Ryzen AI processors can do In this text they didn’t promise Ryzen AI processors could do anything, they promised the Ryzen AI software would let you deploy models to Ryzen AI processors. > The issue is with the limited understanding you've taken on for your argument, you didn't realise that all computers can already do what they promised Ryzen AI processors can do. I have deployed machine learning models to the edge in production on accelerators and CPUs. I use Llamas on my home rig, both CPU and CUDA. I am quite aware of what they can do, which is why I already knew the performance of the Ryzen AI processor. You are the one whose limited understanding kept you from recognizing that “Ryzen AI processor” is a distinct thing from a CPU or GPU. Your complaint is the equivalent of complaining that the home page for CUDA’s documentation doesn’t explain what a GPU is and why it is different from a CPU, which can do the same computations. You also seem moderately confused about the use cases for these relatively low performance, but very low power ML accelerators. The first thing that shipped on Ryzen’s AI accelerator is a model from Microsoft that makes it look like you are making eye contact with a camera. Not a giant LLM. They don’t compete with GPUs.

daemonologist 3y ago

For comparison, Google's Edge TPU (found in the Coral USB accelerator for example) will do 4 INT8 TOPS [0], an Nvidia T4 will do 130 [1], and an A100 or A6000 will do 620 [2]. Fully utilized it could be expected to be radically faster and more efficient than CPU but still of course much slower than workstation/server hardware for these operations. [0] https://coral.ai/products/accelerator https://coral.ai/products/accelerator [1] https://semiengineering.com/tops-memory-throughput-and-inference-efficiency/ https://semiengineering.com/tops-memory-throughput-and-infer... [2] https://www.microway.com/knowledge-center-articles/in-depth-comparison-of-nvidia-ampere-gpu-accelerators/ https://www.microway.com/knowledge-center-articles/in-depth-...