13 ms·
Ask HN: Is Apple's R1 a Discrete GPU?
I think the R1 in Apple's Vision Pro is a discrete GPU, which would be significant since it would be the first time Apple has shipped a discrete GPU of their own design and could be an indicator that discrete GPUs in Macs are imminent. But I'm asking here because I haven't been able to find any knowledgeable coverage or speculation about what the R1 chip actually is.
The confirmed details of the R1 chip are scant:
- It does a lot of image processing and sensor data integration: "the brand-new R1 chip processes input from 12 cameras, five sensors, and six microphones to ensure that content feels like it is appearing right in front of the user's eyes, in real time. R1 streams new images to the displays within 12 milliseconds"[0]
- It has substantial memory bandwidth: "256GB/s memory bandwidth"[1]
- It uses special, high bandwidth memory: "To support R1’s high-speed processing, SK hynix has developed the custom 1-gigabit DRAM. The new DRAM is known to have increased the number of input and output pins by eightfold to minimize delays. Such chips are also called Low Latency Wide IO. According to experts, the new chip also appears to have been designed using a special packaging method – Fan-Out Wafer Level Packaging – to be attached to the R1 chipset as a single unit […]"[2]
- This is subjective, but: Apple shows it as being roughly the same size as the M2 processor in the Vision Pro marketing[3], indicating it's a peer to the M2. Now, that may mean nothing, but based on my experience with Apple kremlinology they are not arbitrary about stuff like that.
So I see all this and get major GPU vibes. But I'm just some guy on a the internet, so what do I know?
0: https://www.macrumors.com/2023/06/05/apple-reveals-vision-pro-headset/
1: https://www.apple.com/apple-vision-pro/specs/
2: https://9to5mac.com/2023/07/11/vision-pro-performance/
3: https://www.apple.com/apple-vision-pro/
- rowanG077 3y agoI doubt it's a discrete GPU. It doesn't make sense to split a GPU and CPU in such a tiny device. Especially with the gains you can get with completely shared memory. I think it's much more likely to be a SoC with dsp tiles for the sensors and a huge GPU block and standard apple p and e cores.
- deleted 3y ago[deleted]
- rmorey 3y agoI'm very interested in it as well, I think we can mostly just speculate. I don't think it's a GPU, I think it's an independent SOC running some sort of real-time OS (https://en.wikipedia.org/wiki/Real-time_operating_system https://en.wikipedia.org/wiki/Real-time_operating_system) I suppose it would necessarily have GPU cores in it to do image processing, but I don't think it itself is just a GPU, or gives any indication about discrete GPU's coming to the Mac. I think this is a very Vision-specific piece of silicon. Edit: many are saying that is just an Image Signal Processor (https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor). I don't think that's quite the case because 1) The M-series chips are already known to have ISP's packaged into them. and 2) My understanding is that the R1's job is to provide continuity of passthrough even in the event of a kernel panic by the M-series chip. To my thinking, this means the R1 chip must have a level of independence beyond that of a traditional coprocessor. I think it is an entire SOC.
- nicolas_17 3y agoHere we go, kernel panic in the M-series chip: https://twitter.com/0xjprx/status/1753575170101461266 https://twitter.com/0xjprx/status/1753575170101461266
- toasterlovin 3y agoAlright, here's an idea for you: Vision Pro is shipping with the previous generation CPU. What if it was meant to ship earlier, but Apple's discrete GPU efforts got delayed? That could also explain why the M2 Mac Pro shipped without a discrete GPU, which is a glaring omission for that product.
- rmorey 3y agoI don't think you should hold your breath for discrete GPUs coming back any time soon: https://www.owc.com/blog/why-the-m2-ultra-mac-pro-doesnt-support-discrete-gpus https://www.owc.com/blog/why-the-m2-ultra-mac-pro-doesnt-sup...
- toasterlovin 3y ago
- agnokapathetic 3y agoit’s a discrete Image Processing Unit https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor
- rmorey 3y agoI don't think this is correct. The M-series chips themselves have ISP's in them.
- cududa 3y ago“ Apple shows it as being roughly the same size as the M2 processor in the Vision Pro marketing[3], indicating it's a peer to the M2” Errr, that’s not how this works. The size of them being similar means absolutely nothing. The R1 is a coprocessor, I mean it’s basically a big signal processor with a decent amount of RAM to handle all the camera inputs, and does some GPU like stuff, but isn’t a GPU
- toasterlovin 3y agoThe R1 has a lot of memory bandwidth. According to this article at Tom's Hardware[0], the M2 has 68 GB/s of memory bandwidth. So the R1 has several times that at 256GB/s. Coincidentally (or not), that is similar to the discrete Nvidia and AMD GPUs they compare the M2 against in that article. I know discrete GPU comparable memory bandwidth doesn't necessarily mean it's discrete GPU or a peer to the M2, but there's clearly a lot of something going on. Plus isn't it kinda weird to specifically mention a spec that is applicable to programmable devices like CPUs or GPUs on a signal processor? Like, who cares? 0: https://www.tomshardware.com/news/apple-m2-gpu-analysis https://www.tomshardware.com/news/apple-m2-gpu-analysis
- coder543 3y agoM1 was 68GB/s (and the article says this too). M2 was 100GB/s.
- toasterlovin 3y agoYou're right, my mistake.
- snom380 3y agoApple likes mentioning whatever is a competitive advantage. M2 has 100GB/s, M2 Pro 200GB/s. They put in the bandwidth they think is needed for the purpose. Likewise with the 12 camera interfaces. It’s a safe bet that there’s some sort of GPU on board but there’s zero indication that this is Apples first discrete GPU. If anything you would expect such a chip to appear first in a desktop product. The R1 is likely programmable as well.
- choilive 3y agoNo, likely a dedicated DSP chip to handle the aforementioned 12 cameras + 5 sensors + 6 mics.
- bhouston 3y agoI don't think it is a GPU, rather it is probably very similar to a DSP/Neural Network chip. Lots of matrix multiplication capabilities and streaming data. https://en.wikipedia.org/wiki/Digital_signal_processor https://en.wikipedia.org/wiki/Digital_signal_processor (Of which https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor are a subset of, which someone else in this forum mentions.) DSPs are similar to GPUs in many respects but they are also similar to CPUs in many respects, but it doesn't make them the same.
- toasterlovin 3y agoA few points: 1. GPUs are good at matrix multiplication! 2. GPUs are programmable, so you can do software updates, which seems like it would be useful if they're also using the R1 to do object recognition and other machine vision tasks. 3. GPUs would be useful in other parts of their product lineup.
- snom380 3y agoLots of processor types are programmable, not just GPUs. Apple is well known for making purpose built chips for their hardware. Main point is that they reuse the cores across different chips. A chip with 12 camera interfaces doesn’t sound like a general purpose GPU to me.
- toasterlovin 3y agoJust did some reading on image signal processors and then noticed that Apple calls out that the A series chips do ISP on-die rather than via a dedicated co-processor, so it seems like you’re right and a significant part of this chip is just to be a DSP of raw sensor input. I guess the unknown is what exactly happens on this chip vs the M2. Clearly some computer vision is being done to do object/people recognition and to separate out objects in order to place the content in 3D space. And then there’s stuff like deciding when to allow the external environment to “break through” immersive content. And of course placing the content in the scene. It seems like those would make the most sense on the R1, since it avoids contention with “userland” processes. But I don’t know if those tasks are more GPU tasks or DSP tasks. If more GPU, then I guess this thing is a SoC with a lot of die space dedicated to DSP.
- polyomino 3y agoIt's basically Apple's version of Microsoft's Holographic Processing Unit
- CharlesW 3y agoThe R1 is an SPU (Sensor Processing Unit) which works in concert with the M2's integrated GPU (the R1 being the innie, and the M2 being the outie). It is doing an enormous amount of image signal processing, which seems to explain the 256GB/s memory bandwidth. https://techcrunch.com/2023/06/05/apple-r1-chip-apple-vision-pro/ https://techcrunch.com/2023/06/05/apple-r1-chip-apple-vision...: "The specialized chip was designed specifically for the challenging task of real-time sensor processing, taking the input from 12 cameras, five sensors (including a lidar sensor!) and six microphones. The company claims it can process the sensor data within 12 milliseconds — eight times faster than the blink of an eye — and says this will dramatically reduce the motion sickness plaguing many other AR/VR systems."
- giantrobot 3y agoI think this is the best answer. The R1 is directly connected to the device's sensors and can write out state to the shared main memory. Processing the sensor state in 12ms gives 4ms to consume the world state and draw a video frame and hit a solid 60fps. Any lag from the position of your head/body and the eye display is going to mess with your proprioception. The worse that lag the more likely you are to get motion sickness.
- halJordan 3y agoApple's claim is "photon to photon" for the 12ms timeline, iirc. Which jives better with the actual frame rate of the headset.
- toasterlovin 3y ago> and can write out state to the shared main memory I think it would actually make more sense for the M1 to treat the R1 as a display that it writes final composited frames to, then the R1 integrates the output from the M1 into the rest of the scene it’s rendering from the other sensors. IE, the output of the M1 is essentially another camera input to the R1 (well, camera plus multi-channel audio).
- jazzyjackson 3y ago60hz is great while looking at a screen but may not be enough for keeping a good illusion going while you move around space. hololens has a similar setup that renders triangles at whatever rate it can but updates the position on the display at 240hz (that is, even if the next frame of an animation is lagging, its position in space will be adjusted as you move) imo the illusion was rock solid, extremely challenging given that the display was transparent so you had to keep up with the real backdrop moving - vision pro and all passthrough devices get to fake it but at the cost of proprioception as you said
- wtallis 3y agoA 1 Gigabit DRAM chip sounds far smaller than anything that's been used with GPUs in a very long time. That's a capacity from 15 years ago.
- utopcell 3y agothey probably don't use a single chip.
- wtallis 3y agoThat doesn't alleviate the capacity issue. Even if it's a 1Gbit chip with an 8-bit interface, then a 256-bit bus would only add up to 4 GB of DRAM at most. The ratio of capacity to bandwidth is just not anywhere close to what makes sense for ordinary GPU usage.
- utopcell 3y agoThis doesn't seem to be an ordinary GPU however. It needs to process input from all the sensors and produce some low-bw output (positioning, object detection from lidar etc). They probably have enough RAM to buffer/post-process whatever they need from the sensors.
- phire 3y agoThe capacity might be small, but with 256GB/s of bandwidth, that's right up there at modern GPU levels, roughly comparable to a RTX 4060. And that's apparently from single chip, the RTX 4060 needs 6 chips get equivalent bandwidth. Though for that exact reason, I'm really not sure I trust the "single chip" part of the rumour. That's a lot of bandwidth per chip and it seems more likely to have four memory chips, which would result in 2 GB of capacity.
- nicolas_17 3y agoMaybe instead of speculating based on Apple's marketing and based on articles from other people speculating, we can make some guesses from actual data. Here's an obviously-incomplete list of source files part of the R1 firmware, probably referenced from asserts or other logging messages, thus present as strings in the firmware binary: https://transfer.archivete.am/inline/Ydfxb/bora.txt https://transfer.archivete.am/inline/Ydfxb/bora.txt It seems it's handling data from cameras (CImageSensor*), LIDAR (SensorMgr/Tof = time of flight?), and display (DCP). I also see mentions of accel, gyro, bmi284 (IMU from Bosch?).
- phire 3y agoInteresting. The H13 codename suggests it's actually M1 derived. And no sign of the GPU, which goes by the name AGX, seems to be more IPU than anything else.
- _kulang 3y agoI thought it was fairly clear from Apple’s comments that it is essentially single function hardware that does all of the processing for AR/VR. I mean, they say that it is all done inside 12ms. There’s no way that it is just a GPGPU. Modern GPGPUs, particularly the NVIDIA stack, have a bunch of single function units internally which do things like optical flow on the image buffers etc. but similarity sort of stops there You can do a lot with not much if it is all specialised hardware. Some of the wider features of the chip are due to the huge data bandwidth. But general purpose processing is a little slow for this task, certainly at this power envelope
- amelius 3y agoIt's proprietary magical undocumented locked-down tech that I don't want anything to do with, regardless of how well it performs.
- phire 3y agoThere is no way Apple designed the R1 from scratch, it will contain a bunch of components from their other SoC designs. Obviously it's going to have quite a few more IPUs than a standard SoC, which only have to handle ~4 cameras. It also clearly has the display scan-out components. And it's entirely possible that one of the components Apple took from their grap-bag of SoC component is the GPU compute cores. Properly not the rasterizer or texture samplers, but I could see the compute cores being useful for running tracking algorithms. The massive amount of memory bandwidth does kind of suggest GPU compute cores. But even if it does, that doesn't make it a discrete GPU. Just a dedicated SoC with some GPU components, and a bunch of other things. But it simply can't be used as in indicator of discrete GPUs in Macs, because all those other SoC components would be a waste of silicon and it's unlikely Apple would reuse the R1 die as a dedicated GPU.
- toasterlovin 3y ago> But even if it does, that doesn't make it a discrete GPU. Just a dedicated SoC with some GPU components, and a bunch of other things. Yeah, this is where I'm at after reading all the comments on this thread.