7 ms·

Some of the main assertions about how common GPUs work are totally backwards: "when a GPU is waiting for data, it can't just switch to work on something else"

Some of the main assertions about how common GPUs work are totally backwards: "when a GPU is waiting for data, it can't just switch to work on something else" - GPU designs specifically use the idea of rotating between threads to not block on memory access. "the concept that there is a local on-die memory pool you can read/write from with very very low perf impact is unthinkable in the current desktop GPU space" - Optimizing for the characteristics of the specific device's memory hierarchy including local/shared memory is a big part of GPU coding. If Apple follows the pattern that companies like NVIDIA have used they will keep the overall shape of the GPU similar between generations (in terms of how many threads, memory hierarchy, etc). This will give app devs relatively similar + stable tuning targets, we'll see high dollar apps get aggressively optimized for specific chips, and as that happens power/heat will go up. For an extreme example of this maturation curve look at how console games get more and more out of the same hardware. All that said the Metal tile shading API is a real thing Apple has been pushing for years on the mobile side, presumably because it fits the hardware and performs better, and now Macs use similar chips. No surprises there. And if an app is built around OpenGL and now needs to port to Metal to improve performance, sure, that could be a lot of work.

SwordOfMyBone 5y ago
Don't know anything about GPUs and am a CS undergrad. How does this apply to compute workloads? What's wrong with treating it like an IMR GPU when it comes to compute? is the tile buffer used for compute? ELI 5 please.
- Maursault 5y ago
  > Don't know anything about GPUs and am a CS undergrad. You are the computer in "Computer Science," and the science is really just a lot of math. Maybe you wanted to study CE instead? I made the same damn mistake.
  - SwordOfMyBone 5y ago
    I unironically enjoy CS. I like programming and I like being close to the metal. I have an interest in the engineering part but I don't know if I'd pick it over CS. Life is hard. Also like compilers, and most compiler engineers are CS grads. That's my dream job!
    - Maursault 5y ago
      And yet, ironically, Computer Science has absolutely nothing to do with programming or computers. CS is a subset of Mathematics. That's all it is. I was being literal when I claimed you were the computer, as in the one who computes. This is confusing, and I am not the only one that would like to change its name to Reckoning Science, as no one would misunderstand what that means. A computer scientist can solve very hard problems. But a computer engineer can also solve hard problems, yet they can also solve really easy ones, giving them the edge.
      - butlerm 5y ago
        I believe you are being unnecessarily pedantic. Real world computer science programs cover software and hardware architecture as a matter of course. If they didn't 90% of them could be shut down for lack of interest.
        Maursault 5y ago
        You are mistaken about the curriculum, not that what you've said is entirely false, but that you are incorrectly attributing too much weight to these things in CS. Countless centuries before software or hardware existed, there were computer scientists. How could that be possible? Because software development is not computer science, nor is study of hardware architecture. A decent CS program will include a some amount of history as well, who first did what and when, why the thing is called what it is, etc. Does that mean historians are computer scientists?! No, a university will round out a curriculum to include germane information that isn't really considered an essential element of that curriculum. In CS, the math is critical. So we can remove the computers and remove programming from the CS curriculum and still have a computer science program. But without the math, there is no CS.
  - coldtea 5y ago
    Someone still needs to know the math too.
  - anonymoushn 5y ago
    It seems like it would be good if people who join to ask a question do not receive answers such as this.
- hedgehog 5y ago
  I'm not familiar with the tile-specific stuff Apple is doing but I think the grain of truth in that thread is some graphics apps on Apple silicon will perform better if they use them. The rest of the thread is doesn't make sense to me. If you want an idea of what GPU programming is like you could look at some CUDA intros: - https://developer.nvidia.com/blog/cuda-refresher-cuda-programming-model/ https://developer.nvidia.com/blog/cuda-refresher-cuda-progra... - http://cuda.ce.rit.edu/cuda_overview/cuda_overview.htm http://cuda.ce.rit.edu/cuda_overview/cuda_overview.htm If you don't have CUDA hardware you can get various versions of OpenCL that will at least work on your CPU. Lots of interesting compiler stuff happening targeting GPUs.
- ribit 5y ago
  There is nothing wrong with treating G13 as an IMR GPU. But might be able better performance and efficiency when doing for example image processing (or a task that maps to it conceptually) via the rendering pipeline with tile shaders. This way you can process your data in small chunks, using very fast on-GPU memory file. In the end, tile memory is the same thing as shared memory in CUDA, but you can persist its contents between multiple different compute and fragment shaders. It's a very powerful programming model that makes local data relationships explicit.
ribit 5y ago
Quick comment on the tile stuff: Apple GPUs basically rasterize primitives into small tiles (16x16 or 32x32 pixels), tracking primitive coverage and barycentric coordinates per pixel. Once all primitives are rasterized, the entire tile is shaded at once. That’s where the “deferred” in tile-based deferred shading comes from. This helps to utilize both the compute hardware and memory subsystem more efficiently. Tile data is stored in the very fast on-chip shared memory (essentially large programmable register file). Apple also exposes this abstraction to the user, allowing them to freely manipulate the contents of this memory and persist it through multiple kernel invocations. Using tile memory explicitly can radically simplify some rendering algorithms, improve performance efficiency and reduce the RAM bandwidth needs. Since the concept tiles are so important to Apple GPUs, they also support bulk memory moves which allow one to push the contents of the entire tile (multiple KBs) in one operation, which is obviously more efficient than doing scattered writes. So it’s exactly as you say: of course one can improve performance by using these features, but it’s not that one has to do it to get good performance in the first place. If there are performance issues with the Ultra, memory hierarchy is an unlikely culprit IMO.

SwordOfMyBone 5y ago

Don't know anything about GPUs and am a CS undergrad. How does this apply to compute workloads? What's wrong with treating it like an IMR GPU when it comes to compute? is the tile buffer used for compute? ELI 5 please.

Maursault 5y ago
> Don't know anything about GPUs and am a CS undergrad. You are the computer in "Computer Science," and the science is really just a lot of math. Maybe you wanted to study CE instead? I made the same damn mistake.
- SwordOfMyBone 5y ago
  I unironically enjoy CS. I like programming and I like being close to the metal. I have an interest in the engineering part but I don't know if I'd pick it over CS. Life is hard. Also like compilers, and most compiler engineers are CS grads. That's my dream job!
  - Maursault 5y ago
    And yet, ironically, Computer Science has absolutely nothing to do with programming or computers. CS is a subset of Mathematics. That's all it is. I was being literal when I claimed you were the computer, as in the one who computes. This is confusing, and I am not the only one that would like to change its name to Reckoning Science, as no one would misunderstand what that means. A computer scientist can solve very hard problems. But a computer engineer can also solve hard problems, yet they can also solve really easy ones, giving them the edge.
    - butlerm 5y ago
      I believe you are being unnecessarily pedantic. Real world computer science programs cover software and hardware architecture as a matter of course. If they didn't 90% of them could be shut down for lack of interest.
      - Maursault 5y ago
        You are mistaken about the curriculum, not that what you've said is entirely false, but that you are incorrectly attributing too much weight to these things in CS. Countless centuries before software or hardware existed, there were computer scientists. How could that be possible? Because software development is not computer science, nor is study of hardware architecture. A decent CS program will include a some amount of history as well, who first did what and when, why the thing is called what it is, etc. Does that mean historians are computer scientists?! No, a university will round out a curriculum to include germane information that isn't really considered an essential element of that curriculum. In CS, the math is critical. So we can remove the computers and remove programming from the CS curriculum and still have a computer science program. But without the math, there is no CS.
- coldtea 5y ago
  Someone still needs to know the math too.
- anonymoushn 5y ago
  It seems like it would be good if people who join to ask a question do not receive answers such as this.
hedgehog 5y ago
I'm not familiar with the tile-specific stuff Apple is doing but I think the grain of truth in that thread is some graphics apps on Apple silicon will perform better if they use them. The rest of the thread is doesn't make sense to me. If you want an idea of what GPU programming is like you could look at some CUDA intros: - https://developer.nvidia.com/blog/cuda-refresher-cuda-programming-model/ https://developer.nvidia.com/blog/cuda-refresher-cuda-progra... - http://cuda.ce.rit.edu/cuda_overview/cuda_overview.htm http://cuda.ce.rit.edu/cuda_overview/cuda_overview.htm If you don't have CUDA hardware you can get various versions of OpenCL that will at least work on your CPU. Lots of interesting compiler stuff happening targeting GPUs.
ribit 5y ago
There is nothing wrong with treating G13 as an IMR GPU. But might be able better performance and efficiency when doing for example image processing (or a task that maps to it conceptually) via the rendering pipeline with tile shaders. This way you can process your data in small chunks, using very fast on-GPU memory file. In the end, tile memory is the same thing as shared memory in CUDA, but you can persist its contents between multiple different compute and fragment shaders. It's a very powerful programming model that makes local data relationships explicit.

ribit 5y ago

Quick comment on the tile stuff: Apple GPUs basically rasterize primitives into small tiles (16x16 or 32x32 pixels), tracking primitive coverage and barycentric coordinates per pixel. Once all primitives are rasterized, the entire tile is shaded at once. That’s where the “deferred” in tile-based deferred shading comes from. This helps to utilize both the compute hardware and memory subsystem more efficiently. Tile data is stored in the very fast on-chip shared memory (essentially large programmable register file). Apple also exposes this abstraction to the user, allowing them to freely manipulate the contents of this memory and persist it through multiple kernel invocations. Using tile memory explicitly can radically simplify some rendering algorithms, improve performance efficiency and reduce the RAM bandwidth needs. Since the concept tiles are so important to Apple GPUs, they also support bulk memory moves which allow one to push the contents of the entire tile (multiple KBs) in one operation, which is obviously more efficient than doing scattered writes. So it’s exactly as you say: of course one can improve performance by using these features, but it’s not that one has to do it to get good performance in the first place. If there are performance issues with the Ultra, memory hierarchy is an unlikely culprit IMO.