4 ms·

> Something I’ve never understood, if these effects are to run in a game, isn’t the GPU already busy? Short answer: No, it's not "already busy". GPU's are so p

> Something I’ve never understood, if these effects are to run in a game, isn’t the GPU already busy? Short answer: No, it's not "already busy". GPU's are so powerful now that you can do physics, fancy render passes, fluid sims, "Game AI" unit pathing, and more, at 100+ FPS. Long answer: You have a "frame budget" which is the amount of time between rendering the super fast "slide show" of frames at 60+ FPS. This gives you between 5 and 30 ms to do a bunch of computation to get the results you need to compute state and render the next frame. That could be moving units around a map, calculating fire physics, blitting terrain textures, rendering verts with materials. In many game engines, you will see a GPU doing dozens of these separate computations per frame. GPU's are basically just a secondary computer attached to your main computer. You give it a bunch of jobs to do every frame and it outputs the results. You combine results into something that looks like a game. > Can this stuff run on an iGPU while the dGPU is doing more rendering-related tasks? Almost no one is using the iGPU for anything. It's completely ignored because it's usually completely useless compared to your main discrete GPU.

bee_rider 3y ago
It looks like the GPU is doing most of the work… from that point of view when do we start to wonder if the GPU can “offload” anything to the whole computer that is hanging off of it, haha.
- softfalcon 3y ago
  > It looks like the GPU is doing most of the work Yes. The GPU is doing most of the work in a lot of modern games. It isn't great at everything though, and there are limitations due to its architecture being structured almost solely for the purpose of computing massively parallel instructions. > when do we start to wonder if the GPU can “offload” anything to the whole computer that is hanging off of it The main bottleneck for speed on most teams is not having enough "GPU devs" to move stuff off the CPU and onto the GPU. Many games suffer in performance due to folks not knowing how to use the GPU properly. Because of this, nVidia/AMD invest heavily in making general purpose compute easier and easier on the GPU. The successes they have had in doing this over the last decade are nothing less than staggering. Ultimately, the way it's looking, GPU's are trying to become good at everything the CPU does and then some. We already have modern cloud server architectures that are 90% GPU and 10% CPU as a complete SoC. Eventually, the CPU may cease to exist entirely as its fundamental design becomes obsolete. This is usually called a GPGPU in modern server infrastructure.
  - vlovich123 3y ago
    I’m pretty sure CPUs destroy GPUs at sequential programming and most programs are written in a sequential style. Not sure where the 90/10 claim comes from but there’s plenty of cloud servers with no GPU installed whatsoever and 0 servers without a CPU.
    - softfalcon 3y ago
      Yup, and until we get a truly general purpose compute GPU that can handle both styles of instruction with automated multi-threading and state management, this will continue. What I've seen shows me that nVidia is working very hard to eliminate this gap though. General purpose computing on the GPU has never been easier, and it gets better every year. In my opinion, it's only a matter of time before we can run anything we want on the GPU and realize various speed gains. As for where the 90/10 comes from, it's from the emerging architectures for advanced AI/graphics compute like the DGX H100 [0]. [0] https://www.nvidia.com/en-us/data-center/dgx-h100/ https://www.nvidia.com/en-us/data-center/dgx-h100/
      - vlovich123 3y ago
        AI is different. Those servers are set up to run AI jobs & nothing else. That’s still a small fraction of overall cloud machines at the moment. Even if in volume they overtake, that’s just because of the huge surge in demand for AI * the compute requirements associated with it eclipsing the compute requirements for “traditional” cloud compute that is used to keep businesses running. I don’t think you’ll see GPUs running things like databases or the Linux kernel. GPUs may even come with embedded ARM CPUs to run the kernel & only run AI tasks as part of the package as a cost reduction, but I think that’ll take a very long time because you have to figure out how to do cotenancy. It’ll depend on if the CPU remains a huge unnecessary cost for AI servers. I doubt that GPUs will get much better at sequential tasks because it’s an essential programming tradeoff (e.g. it’s the same reason you don’t see everything written in SIMD as SIMD is much closer to GPU-style programming than the more general sequential style)
  - kqr 3y ago
- TillE 3y ago
  A game is more than just rendering, and modern games will absolutely get bottlenecked on lower-end CPUs well before you reach say 144 fps. GamersNexus has done a bunch of videos on the topic.
  - softfalcon 3y ago
    You are not wrong that there are many games that are bottle-necked on lower end CPUs. I would argue that for many CPU bound games, they could find better ways to utilize the GPU for computation and it is likely they just didn't have the knowledge, time, or budget to do so. It's easier to write CPU code, every programmer can do it, so it's the most often reached for tool. Also, at high frame rates, the bottleneck is frequently the CPU due to it not feeding the GPU fast enough, so you lose frames. There is definitely a real world requirement of having a fast enough CPU to properly utilize a high end video card, even if it's just for shoving command buffers and nothing else.
vlovich123 3y ago
> No one is using the iGPU for anything. It's completely ignored because it's usually completely useless compared to your main discrete GPU. Modern iGPUs are actually quite powerful is my understanding. I think the reason no one does this is that the software model isn’t actually there/standardized/able to work cross vendor since the iGPU and the discrete card are going to be different vendors typically. There’s also little motivation to do this because not everyone has an iGPU which dilutes the economy of scale of using it. It would be a neat idea to try to run lighter weight things on the iGPU to free up rendering time on the dGPU and make frame rates more consistent, but the incentives aren’t there.
- softfalcon 3y ago
  I agree the incentives aren't there. Also agree that it is possible to use the integrated GPU for light tasks, but only light tasks. In the high performance scenarios where there is all three (discrete GPU, integrated GPU, and CPU) and we try and use the integrated GPU alongside the CPU, it often causes thermal throttling on the shared die between iGPU and CPU. This slows the CPU down from executing well, keeping up with state changes, and sending needed data to the keep the discrete GPU utilized. In short, don't warm up the CPU, we want it to stay cool, if that means not doing iGPU stuff, don't do it. When we have multiple discrete GPU's available (render farm), this on-die thermal bottleneck goes away and there are many render pipelines that are made to handle hundreds, even thousands of simultaneous GPU's working on a shared problem set of diverse tasks, similar to trying to utilize both iGPU and dGPU on the same machine but bigger. Whether or not to use the iGPU is less about scheduling and more about thermal throttling.
  - vlovich123 3y ago
    That’s probably a better point as to why it’s not invested in although most games are not CPU bound so thermal throttling wouldn’t apply then as much. I think it’s multiple factors combined. The render pipelines you refer to are all offline non-realtime rendering though for movies/animation/etc right? Somewhat different UX and problem space than realtime gaming.
- zokier 3y ago
johnnyanmac 3y ago
In theory it's perfectly possible to do all you describe in 8ms (i.e. VR render times). In reality we're 1. still far from properly utilizing modern graphics APIs as is. Some of the largest studios are close but knowledge is tight lipped in the industry. 2. even when those top studios can/do, they choose to focus more of the budget on higher render resolution over adding more logic or simulation. Makes for superficially better looking games to help sell. 3. and of course there are other expensive factors right now with more attention like Ray traced lighting which can only be optimized so much on current hardware. I'd really love to see what the AA or maybe even indie market can do with such techniques one day. I don't have much faith that AAA studios will ever prioritize simulation.
- mrcode007 3y ago
  Simulation is often already done. Except it’s done offline and then replayed in the game over and over.
- softfalcon 3y ago
  If you enjoy simulation on that level, you might find Star Citizen [0] or the latest updates to No Man's Sky [1] interesting. You're right that simulation is often an after thought. Most games prioritize narrative, combat, artist's vision and high fidelity environments over complex systems you can interact with. There are a few outliers though. Alternatively, you get so far into the simulation side of things with stuff like Dwarf Fortress [2] and all visual fidelity is thrown away for the sake of prioritizing simulation complexity! [0] https://robertsspaceindustries.com/ https://robertsspaceindustries.com/ [1] https://www.nomanssky.com/ https://www.nomanssky.com/ [2] https://www.bay12games.com/dwarves/ https://www.bay12games.com/dwarves/
smolder 3y ago
I actually have my desktop AMD iGPU enabled despite using a discrete Nvidia card. I use the iGPU to do AI noise reduction for my mic in voice and video calls. I'm not sure if this is really an ideal setup, having both GPUs enabled with both driver packages installed (as compared to just running Nvidias noise reduction on the dGPU, I guess,) but it seems to all work with no issue. The onboard HDMI can even be used at the same time as the discrete cards monitor ports.

bee_rider 3y ago

It looks like the GPU is doing most of the work… from that point of view when do we start to wonder if the GPU can “offload” anything to the whole computer that is hanging off of it, haha.

softfalcon 3y ago
> It looks like the GPU is doing most of the work Yes. The GPU is doing most of the work in a lot of modern games. It isn't great at everything though, and there are limitations due to its architecture being structured almost solely for the purpose of computing massively parallel instructions. > when do we start to wonder if the GPU can “offload” anything to the whole computer that is hanging off of it The main bottleneck for speed on most teams is not having enough "GPU devs" to move stuff off the CPU and onto the GPU. Many games suffer in performance due to folks not knowing how to use the GPU properly. Because of this, nVidia/AMD invest heavily in making general purpose compute easier and easier on the GPU. The successes they have had in doing this over the last decade are nothing less than staggering. Ultimately, the way it's looking, GPU's are trying to become good at everything the CPU does and then some. We already have modern cloud server architectures that are 90% GPU and 10% CPU as a complete SoC. Eventually, the CPU may cease to exist entirely as its fundamental design becomes obsolete. This is usually called a GPGPU in modern server infrastructure.
- vlovich123 3y ago
  I’m pretty sure CPUs destroy GPUs at sequential programming and most programs are written in a sequential style. Not sure where the 90/10 claim comes from but there’s plenty of cloud servers with no GPU installed whatsoever and 0 servers without a CPU.
  - softfalcon 3y ago
    Yup, and until we get a truly general purpose compute GPU that can handle both styles of instruction with automated multi-threading and state management, this will continue. What I've seen shows me that nVidia is working very hard to eliminate this gap though. General purpose computing on the GPU has never been easier, and it gets better every year. In my opinion, it's only a matter of time before we can run anything we want on the GPU and realize various speed gains. As for where the 90/10 comes from, it's from the emerging architectures for advanced AI/graphics compute like the DGX H100 [0]. [0] https://www.nvidia.com/en-us/data-center/dgx-h100/ https://www.nvidia.com/en-us/data-center/dgx-h100/
    - vlovich123 3y ago
      AI is different. Those servers are set up to run AI jobs & nothing else. That’s still a small fraction of overall cloud machines at the moment. Even if in volume they overtake, that’s just because of the huge surge in demand for AI * the compute requirements associated with it eclipsing the compute requirements for “traditional” cloud compute that is used to keep businesses running. I don’t think you’ll see GPUs running things like databases or the Linux kernel. GPUs may even come with embedded ARM CPUs to run the kernel & only run AI tasks as part of the package as a cost reduction, but I think that’ll take a very long time because you have to figure out how to do cotenancy. It’ll depend on if the CPU remains a huge unnecessary cost for AI servers. I doubt that GPUs will get much better at sequential tasks because it’s an essential programming tradeoff (e.g. it’s the same reason you don’t see everything written in SIMD as SIMD is much closer to GPU-style programming than the more general sequential style)
- kqr 3y ago
TillE 3y ago
A game is more than just rendering, and modern games will absolutely get bottlenecked on lower-end CPUs well before you reach say 144 fps. GamersNexus has done a bunch of videos on the topic.
- softfalcon 3y ago
  You are not wrong that there are many games that are bottle-necked on lower end CPUs. I would argue that for many CPU bound games, they could find better ways to utilize the GPU for computation and it is likely they just didn't have the knowledge, time, or budget to do so. It's easier to write CPU code, every programmer can do it, so it's the most often reached for tool. Also, at high frame rates, the bottleneck is frequently the CPU due to it not feeding the GPU fast enough, so you lose frames. There is definitely a real world requirement of having a fast enough CPU to properly utilize a high end video card, even if it's just for shoving command buffers and nothing else.

vlovich123 3y ago

> No one is using the iGPU for anything. It's completely ignored because it's usually completely useless compared to your main discrete GPU. Modern iGPUs are actually quite powerful is my understanding. I think the reason no one does this is that the software model isn’t actually there/standardized/able to work cross vendor since the iGPU and the discrete card are going to be different vendors typically. There’s also little motivation to do this because not everyone has an iGPU which dilutes the economy of scale of using it. It would be a neat idea to try to run lighter weight things on the iGPU to free up rendering time on the dGPU and make frame rates more consistent, but the incentives aren’t there.

softfalcon 3y ago
I agree the incentives aren't there. Also agree that it is possible to use the integrated GPU for light tasks, but only light tasks. In the high performance scenarios where there is all three (discrete GPU, integrated GPU, and CPU) and we try and use the integrated GPU alongside the CPU, it often causes thermal throttling on the shared die between iGPU and CPU. This slows the CPU down from executing well, keeping up with state changes, and sending needed data to the keep the discrete GPU utilized. In short, don't warm up the CPU, we want it to stay cool, if that means not doing iGPU stuff, don't do it. When we have multiple discrete GPU's available (render farm), this on-die thermal bottleneck goes away and there are many render pipelines that are made to handle hundreds, even thousands of simultaneous GPU's working on a shared problem set of diverse tasks, similar to trying to utilize both iGPU and dGPU on the same machine but bigger. Whether or not to use the iGPU is less about scheduling and more about thermal throttling.
- vlovich123 3y ago
  That’s probably a better point as to why it’s not invested in although most games are not CPU bound so thermal throttling wouldn’t apply then as much. I think it’s multiple factors combined. The render pipelines you refer to are all offline non-realtime rendering though for movies/animation/etc right? Somewhat different UX and problem space than realtime gaming.
zokier 3y ago

johnnyanmac 3y ago

In theory it's perfectly possible to do all you describe in 8ms (i.e. VR render times). In reality we're 1. still far from properly utilizing modern graphics APIs as is. Some of the largest studios are close but knowledge is tight lipped in the industry. 2. even when those top studios can/do, they choose to focus more of the budget on higher render resolution over adding more logic or simulation. Makes for superficially better looking games to help sell. 3. and of course there are other expensive factors right now with more attention like Ray traced lighting which can only be optimized so much on current hardware. I'd really love to see what the AA or maybe even indie market can do with such techniques one day. I don't have much faith that AAA studios will ever prioritize simulation.

mrcode007 3y ago
Simulation is often already done. Except it’s done offline and then replayed in the game over and over.
softfalcon 3y ago
If you enjoy simulation on that level, you might find Star Citizen [0] or the latest updates to No Man's Sky [1] interesting. You're right that simulation is often an after thought. Most games prioritize narrative, combat, artist's vision and high fidelity environments over complex systems you can interact with. There are a few outliers though. Alternatively, you get so far into the simulation side of things with stuff like Dwarf Fortress [2] and all visual fidelity is thrown away for the sake of prioritizing simulation complexity! [0] https://robertsspaceindustries.com/ https://robertsspaceindustries.com/ [1] https://www.nomanssky.com/ https://www.nomanssky.com/ [2] https://www.bay12games.com/dwarves/ https://www.bay12games.com/dwarves/

smolder 3y ago

I actually have my desktop AMD iGPU enabled despite using a discrete Nvidia card. I use the iGPU to do AI noise reduction for my mic in voice and video calls. I'm not sure if this is really an ideal setup, having both GPUs enabled with both driver packages installed (as compared to just running Nvidias noise reduction on the dGPU, I guess,) but it seems to all work with no issue. The onboard HDMI can even be used at the same time as the discrete cards monitor ports.