5 ms·

Now that LLMs run on GPU too, future GPUs will need to juggle between the graphics, the physics and the AI for NPCs. Fun times trying to balance all that. My g

Now that LLMs run on GPU too, future GPUs will need to juggle between the graphics, the physics and the AI for NPCs. Fun times trying to balance all that. My guess is that the load will become more and more shared between local and remote computing resources.

teh_infallible 3y ago
Maybe in the future, personal computers will have more than one GPU, one for graphics and one for AI?
- MaxBarraclough 3y ago
  Many computers already have 2 GPUs, one integrated into the CPU die, and one external (and typically enormously more powerful). To my knowledge though it's very rare for software to take advantage of this.
  - softfalcon 3y ago
    In high performance scenarios, the GPU is running full blast while the CPU is running at full blast just feeding data and pre-process work to the GPU. The GPU is the steam engine hurtling forward, the CPU is just the person shoveling coal into the furnace. Using the integrated GPU heats up the main die where the CPU is because they live together on the same chip. The die heats up, CPU thermal throttles, CPU stops efficiently feeding data to the GPU at max speed, GPU slows down from under utilization. In high performance scenarios, the integrated GPU is often a waste of thermal budget.
    - MaxBarraclough 3y ago
      Doesn't this assume inadequate cooling? A quick google indicates AMD's X3D CPUs begin throttling around 89°C, and that it's not overly challenging to keep them below 80 even under intense CPU load, although that's presumably without any activity on the integrated GPU. Assuming cooling really is inadequate for running both the CPU cores and the integrated GPU: for GPU-friendly workloads (i.e. no GPU-unfriendly preprocessing operations for the CPU) it would surely make more sense to use the integrated GPU rather than spend the thermal budget having the CPU cores do that work.
      - softfalcon 3y ago
        What the specs say they'll do and what they actually do are often very different realities in my experience. I've seen thermal throttling happening at 60°C because overall, the chip is cool, but one core is maxing or two cores are maxing. Which is common in game dev with your primary thread feeding a GPU with command buffer queues and another scheduling the main game loop. Even when water cooled or the high end air cooling on my server blades, I see that long term, the system just hits a trade-off point of ~60-70°C and ~85% max CPU clock even when the cooling system is industry grade, loud as hell, and has an HVAC unit backing it. Probably part of why scale out is so popular to distribute load. When I give real work to any iGPU's on these systems, I see the temps bump 5-10°C and clocks on the CPU cores drop a bit. Could be drivers, could be temp curves, I would think these fancy cooling systems I'm running are performing well though. shrug
- Cthulhu_ 3y ago
  What do you mean in the future? Multiple GPU chips is already pretty common (on the same card or having multiple cards at the same time). Besides, GPUs are massively parallel chips as well, with specialized units for graphics and AI operations.

teh_infallible 3y ago

Maybe in the future, personal computers will have more than one GPU, one for graphics and one for AI?

MaxBarraclough 3y ago
Many computers already have 2 GPUs, one integrated into the CPU die, and one external (and typically enormously more powerful). To my knowledge though it's very rare for software to take advantage of this.
- softfalcon 3y ago
  In high performance scenarios, the GPU is running full blast while the CPU is running at full blast just feeding data and pre-process work to the GPU. The GPU is the steam engine hurtling forward, the CPU is just the person shoveling coal into the furnace. Using the integrated GPU heats up the main die where the CPU is because they live together on the same chip. The die heats up, CPU thermal throttles, CPU stops efficiently feeding data to the GPU at max speed, GPU slows down from under utilization. In high performance scenarios, the integrated GPU is often a waste of thermal budget.
  - MaxBarraclough 3y ago
    Doesn't this assume inadequate cooling? A quick google indicates AMD's X3D CPUs begin throttling around 89°C, and that it's not overly challenging to keep them below 80 even under intense CPU load, although that's presumably without any activity on the integrated GPU. Assuming cooling really is inadequate for running both the CPU cores and the integrated GPU: for GPU-friendly workloads (i.e. no GPU-unfriendly preprocessing operations for the CPU) it would surely make more sense to use the integrated GPU rather than spend the thermal budget having the CPU cores do that work.
    - softfalcon 3y ago
      What the specs say they'll do and what they actually do are often very different realities in my experience. I've seen thermal throttling happening at 60°C because overall, the chip is cool, but one core is maxing or two cores are maxing. Which is common in game dev with your primary thread feeding a GPU with command buffer queues and another scheduling the main game loop. Even when water cooled or the high end air cooling on my server blades, I see that long term, the system just hits a trade-off point of ~60-70°C and ~85% max CPU clock even when the cooling system is industry grade, loud as hell, and has an HVAC unit backing it. Probably part of why scale out is so popular to distribute load. When I give real work to any iGPU's on these systems, I see the temps bump 5-10°C and clocks on the CPU cores drop a bit. Could be drivers, could be temp curves, I would think these fancy cooling systems I'm running are performing well though. shrug
Cthulhu_ 3y ago
What do you mean in the future? Multiple GPU chips is already pretty common (on the same card or having multiple cards at the same time). Besides, GPUs are massively parallel chips as well, with specialized units for graphics and AI operations.