6 ms·
Apple Patent Shows GPU Dynamic Caching Has Been in Development for Years
- skavi 3y agoAnything shipping in a chip has to have been developed for years. And this particular GPU arch was meant to ship last year.
- deleted 3y ago[deleted]
- KingLancelot 3y ago[dead]
- baggy_trough 3y agoWould love to read a good explanation of what this is.
- jb1991 3y ago[flagged]
- slaymaker1907 3y agoGiven what I know about CUDA and assuming it's different from being able to call malloc within a kernel and paging memory between the CPU and GPU, I assume it's adding in some smartness for offloading stuff from the GPU when running multiple kernels. You want to keep stuff in GPU memory as much as possible since transfers can increase latency dramatically, but if you have multiple applications using the GPU then there might not be enough memory for all of them. However, I'm always suspicious of how innovative patents actually are so it wouldn't surprise me to find out it's just barely different than what CUDA or unified memory on game consoles can do.
- bangonkeyboard 3y agoAll Apple Silicon is unified memory.
- stefanfisk 3y agoWhat about https://www.digitaltrends.com/computing/apple-dynamic-caching-explained/ https://www.digitaltrends.com/computing/apple-dynamic-cachin...?
- tedunangst 3y agoEven after reading that, I'm still uncertain how it lets more blocks/waves/whatev execute. Could really use an example with some fictitious numbers.
- reroute22 3y agoA pretty detailed hypothesis showed up recently: https://forums.macrumors.com/threads/3d-rendering-on-apple-silicon-cpu-gpu.2269416/page-85?post=32694869#post-32694869 https://forums.macrumors.com/threads/3d-rendering-on-apple-s...
- kimixa 3y agoYeah, vague terms like "local memory" aren't helpful - there's lots of different memories that are local to the shader unit that are a shared resource so can affect occupancy (the LDS, register file, TBDR stuff like the tile buffer, and probably any number of other things). Which of these is it referring to? They are separate things in some architectures, but maybe not in all? There's the linked patent itself [0] - but that specifically seems to refer to using the MMU as the part that's doing this dynamic allocation and translation - but the majority of those resources above are "before" address translation, with the MMU often being more at the L2 cache level rather than embedded within the shader clusters themselves. Maybe this isn't the "normal" MMU and instead a simpler address translator specifically for those resources that has been added of for this? Or is it something like the parameter buffer (The block of memory used to store the intermediate data between the tiling & rasterization state, and the pixel shaders). That has been "dynamically allocated" in a similar way since Apple were just taking PowerVR cores directly. And then what is the cost of this re-allocation? If there's a cost to performance to allocating new pages, or (like worse) running out of spare pages, it might mean the (graphics API) user still has to be aware of their resource usage in scheduling shaders, so less of the promise of "Just throw things at the hardware and it'll do things optimally" than people might hope. [0] https://patents.google.com/patent/US20210271606A1/en https://patents.google.com/patent/US20210271606A1/en
- nxobject 3y agoReal World Tech's mailing list will have a good conversation about it at some point, for sure. (Although I know the community isn't primarily about GPU.)
- AaronFriel 3y agoIs macOS Dynamic Caching similar to Windows hardware-accelerated GPU Scheduling[1]? That feature purports improve latency and GPU scheduling efficiency. Is the feature here that the OS is delegating more scheduling to the M3's ASC coprocessor[2] - assuming it's similar to the M1? It seems to me that every explanation of dynamic caching in terms of memory is "wrong" - as seen here and in several articles written by folks more familiar with PC hardware. I think where some folks might get it wrong is thinking of Apple silicon as being like PC hardware, where VRAM and RAM are distinct pools of memory and using the CPU to move data between them (or other devices) is very inefficient. PCI bus attached devices having separate pools of memory has given rise to a plethora of technologies to allow directly read from RAM (DMA), or to enable a GPU to read from NVMe (DirectStorage on Windows), and so on. The patent from the article seems to describe unified memory, part of the M1's architecture, not whatever "Dynamic Caching" is; but I'll admit Apple makes it a bit hard to understand what exactly is the case. There's only one person I'd trust to describe what this feature actually is - I'll wait for Asahi Lina to break down what dynamic caching is and whether this is a hardware or OS feature. [1] https://devblogs.microsoft.com/directx/hardware-accelerated-gpu-scheduling/ https://devblogs.microsoft.com/directx/hardware-accelerated-... [2] https://asahilinux.org/2022/11/tales-of-the-m1-gpu/ https://asahilinux.org/2022/11/tales-of-the-m1-gpu/
- userbinator 3y agoI think where some folks might get it wrong is thinking of Apple silicon as being like PC hardware, where VRAM and RAM are distinct pools of memory and using the CPU to move data between them The patent basically describes using a page table to dynamically map pages of RAM between the GPU and CPU, something that Intel's integrated GPUs have been doing for 20 years.
- userbinator 3y agoApple's GPU Dynamic Caching ensures that cache and memory spaces are dynamically assigned based on the actual needs of different tasks and workloads Isn't this not too different from what Intel DVMT was doing twenty years ago? https://en.wikipedia.org/wiki/Dynamic_video_memory_technology https://en.wikipedia.org/wiki/Dynamic_video_memory_technolog... Then again, this is the same Apple that took "integrated graphics" / "UMA" as a marketing point.
- foota 3y agoI don't think this is correct. My understanding is that this is about more efficiently utilizing the GPUs available resources (e.g., cache) at the core-ish level. Like imagine if hyperthreads could share register space based on what they're doing at the time, I think.
- deleted 3y ago[deleted]
- acchow 3y agoThe OP isn't really about UMA, but about caching. In any case, in Intel's DVMT you have a dedicated memory space for graphics. It is a dynamic amount, but it is still dedicated for graphics. You need to copy data from system memory over to graphics memory for it to be handled. In UMA, no data copying is needed. There's 1 memory space
- kcb 3y agoZero copy shared memory absolutely is available on all modern IGPUs. Do people really think such a low hanging fruit was ignored for the past decades? https://www.intel.com/content/dam/develop/external/us/en/documents/opencl-zero-copy-in-opencl-1-2.pdf https://www.intel.com/content/dam/develop/external/us/en/doc... https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...
- kllrnohj 3y ago> Do people really think such a low hanging fruit was ignored for the past decades? Yes, they do, it's mentioned as some major M1 revolutionary technology all the time. It's utterly absurd, but that's the power of Apple's marketing.
- clemlesne 3y agoHope Asahi team will integrate that soon.
- iAMkenough 3y agoThey have to focus on basic M3 support first. Curious to see if their existing OpenGL driver for M1/M2 will be compatible.
- faitswulff 3y agoThe patent is here: https://patentimages.storage.googleapis.com/2d/72/df/117a4012487c19/US20210271606A1.pdf https://patentimages.storage.googleapis.com/2d/72/df/117a401...
- herniatedeel 3y agoThis patent hasn't issued - it's just a publication.
- ginko 3y ago>Apple patented GPU Dynamic Caching back in 2020. I know this might seem like a lifetime for SW people but <3 years from patent to customer for a GPU HW innovation isn't really that long? It's pretty typical, really.
- Mrirazak1 3y agoI don’t know why we’re surprised about the fact they are working on things for years. It’s how innovation usually works.
- pjmlp 3y agoIt is the usual "overnight success, 15 years in the making" surprise.
- kridsdale3 3y agoBecause a lot of us work in organizations led by Program Managers and directors who expect results within the same financial quarter as the original planning session.
- tibbydudeza 3y agoSo do they have dedicated raytracing units sitting in the pipeline that is not used when the game does not utilize raytracing ???.
- reroute22 3y agoWe don't know anything about Apple's implementation, but for perspective, in Nvidia's first implementation of RTX RT cores occupied about 6-7% of the area of a TPC, where TPCs constitute the majority, but not the entire die, so it's an even smaller (but not by much) portion of the entire die. Maybe 5% or so. It's possible that the latest gen of RT cores are a slightly larger portion of a TPC (but we don't know), but on the other hand, TPCs are a smaller portion of a die now as TPC arrays do not include L2, and L2 got many times bigger. So chances are, RT over die area is still somewhere on the order of 5%. And in Nvidia designs as far as I understand RT cores sit idle in non-RT workloads, yes. If anything, Tensor cores are almost twice as large as RT cores, and tensor cores sit idle whenever you don't use DLSS (or professional ML compute). See https://www.reddit.com/r/hardware/comments/baajes/rtx_adds_195mm2_per_tpc_tensors_125_rt_07/ https://www.reddit.com/r/hardware/comments/baajes/rtx_adds_1...