3 ms·

I don't think it is a GPU, rather it is probably very similar to a DSP/Neural Network chip. Lots of matrix multiplication capabilities and streaming data. htt

I don't think it is a GPU, rather it is probably very similar to a DSP/Neural Network chip. Lots of matrix multiplication capabilities and streaming data. https://en.wikipedia.org/wiki/Digital_signal_processor https://en.wikipedia.org/wiki/Digital_signal_processor (Of which https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor are a subset of, which someone else in this forum mentions.) DSPs are similar to GPUs in many respects but they are also similar to CPUs in many respects, but it doesn't make them the same.

toasterlovin 3y ago
A few points: 1. GPUs are good at matrix multiplication! 2. GPUs are programmable, so you can do software updates, which seems like it would be useful if they're also using the R1 to do object recognition and other machine vision tasks. 3. GPUs would be useful in other parts of their product lineup.
- snom380 3y ago
  Lots of processor types are programmable, not just GPUs. Apple is well known for making purpose built chips for their hardware. Main point is that they reuse the cores across different chips. A chip with 12 camera interfaces doesn’t sound like a general purpose GPU to me.
  - toasterlovin 3y ago
    Just did some reading on image signal processors and then noticed that Apple calls out that the A series chips do ISP on-die rather than via a dedicated co-processor, so it seems like you’re right and a significant part of this chip is just to be a DSP of raw sensor input. I guess the unknown is what exactly happens on this chip vs the M2. Clearly some computer vision is being done to do object/people recognition and to separate out objects in order to place the content in 3D space. And then there’s stuff like deciding when to allow the external environment to “break through” immersive content. And of course placing the content in the scene. It seems like those would make the most sense on the R1, since it avoids contention with “userland” processes. But I don’t know if those tasks are more GPU tasks or DSP tasks. If more GPU, then I guess this thing is a SoC with a lot of die space dedicated to DSP.

toasterlovin 3y ago

A few points: 1. GPUs are good at matrix multiplication! 2. GPUs are programmable, so you can do software updates, which seems like it would be useful if they're also using the R1 to do object recognition and other machine vision tasks. 3. GPUs would be useful in other parts of their product lineup.

snom380 3y ago
Lots of processor types are programmable, not just GPUs. Apple is well known for making purpose built chips for their hardware. Main point is that they reuse the cores across different chips. A chip with 12 camera interfaces doesn’t sound like a general purpose GPU to me.
- toasterlovin 3y ago
  Just did some reading on image signal processors and then noticed that Apple calls out that the A series chips do ISP on-die rather than via a dedicated co-processor, so it seems like you’re right and a significant part of this chip is just to be a DSP of raw sensor input. I guess the unknown is what exactly happens on this chip vs the M2. Clearly some computer vision is being done to do object/people recognition and to separate out objects in order to place the content in 3D space. And then there’s stuff like deciding when to allow the external environment to “break through” immersive content. And of course placing the content in the scene. It seems like those would make the most sense on the R1, since it avoids contention with “userland” processes. But I don’t know if those tasks are more GPU tasks or DSP tasks. If more GPU, then I guess this thing is a SoC with a lot of die space dedicated to DSP.