2 ms·
Fairly common for even pedestrian discrete GPUs to use over 200W. The ludicrous ones can break 1000W. In a given generation, GPU performance is not far from lin
by AnthonyMouse 4y ago
Fairly common for even pedestrian discrete GPUs to use over 200W. The ludicrous ones can break 1000W. In a given generation, GPU performance is not far from linear in power consumption.
The problem is the existing iGPUs are unnecessarily anemic. Okay, the Radeon RX 6950 XT has 80 CU at 335W. You can't fit that in Socket AM5. But the Radeon RX 6400 has 12 CU at 53W. You could add that to a Ryzen 7 7700X (105W) and still have a lower TDP than a Ryzen 9 7900X (170W). You could double that and add it to a Ryzen 7 7700 (65W) and still be under 170W. You could even add that to the 7950X with the caveat that it has the base clock of the 7945HX when the iGPU is under load.
But the Ryzen 7000 series iGPU doesn't have 24 CU, or 12, it has 2. Intel's have been even slower.
The question is, why bother? 2 CU is plenty to run a monitor if you're not doing anything that benefits from a fast GPU. If you are, 12 CU is still on the low side.
But AI could change things, because then even people who don't play games will have a use for that kind of hardware. Professionals will still pay thousands of dollars for a big hot discrete GPU but there is now a case for having ten times as many CU in every business PC.
It's also starting to make more sense to put the CPU and GPU together because of unified memory. The full version of LLaMA requires ~122GB of RAM. That's petty cash in PC memory but a capital expenditure for GPUs. If AMD wanted to make a big splash, they'd drop a new Threadripper with a high TDP socket, a ton of memory channels, support for RDIMMs and a fast iGPU.
This is the thing that may finally unseat Nvidia on AI, because if every CPU from Intel or AMD starts to include an iGPU that can do AI as fast as a $300+ Nvidia discrete GPU, people are going to make the code run on them.
- dragontamer 4y agoDDR5 is something like 10x slower (or 10% of the bandwidth) of GDDR6 or HBM that's used on GPUs. There's no point going much wider on iGPUs, because they're all throttled by DDR4/DDR5 anyway. The exception: PS5 and XBox both have GDDR (graphics RAM) which is 10x more bandwidth than your typical DDR4/DDR5. Since they are no longer memory-bandwidth constrained, they can go much bigger and remain practical.
- AnthonyMouse 4y ago> DDR5 is something like 10x slower (or 10% of the bandwidth) of GDDR6 or HBM that's used on GPUs. DDR5 is something like 50% of the bandwidth of GDDR6. DDR5 up to 8 GT/s, GDDR6 in the range of 14-21 GT/s. For example, the Radeon RX 6400 has GDDR6 at 16 GT/s. But it also has a 64-bit memory bus, compared to 128-bit for Socket AM5. So they have the same memory bandwidth, given the fastest available DDR5. Higher end GPUs have a wider memory bus than that, but those don't fit in the CPU socket's TDP anyway. It wouldn't prevent the iGPU from having 12+ CU instead of 2. Moreover, higher end CPUs have a wider memory bus than that too. Socket SP5 has 12 memory channels. And a max TDP of 360W. That's as much memory bandwidth as many midrange discrete GPUs, especially anything that would fit in its TDP with a reasonable power budget left to the CPU.
- AnthonyMouse 4y agoThey could also add some HBM as a chiplet to the CPU package. That would provide enough bandwidth to feed a fast iGPU and give the CPU a very nice L4 cache.
- sliken 4y agoMore like 2x. Apple for instance uses lpddr5 for 100, 200, 400, or 800GB/sec. It's mostly bus width, not the memory technology that makes a big difference.
- sliken 4y agoKeep in mind that PC sockets have very little memory bandwidth. Even the fancy new AM5 socket with DDR5 and the "recommended" overclocking to DDR5-6000 leaves you with 96GB/sec. Even the lowly RX 6400 has a dedicated 128GB/sec, which isn't trying to keep any CPU cores happy. Doubling the CUs requires double the bandwidth that brings you up to 256GB/sec, almost as much as the new Intel Xeons (307GB/sec) with 8 channels of DDR5-4800 memory. So basically this is why Intel/AMD iGPUs stink, and why Apple's do not. Apple give you a choice of 100, 200, and 400GB/sec in a laptop and 400 or 800GB/sec in a desktop.
- AnthonyMouse 4y agoDDR5-8000 is already available. DDR5-6000 is 75% as fast, which is not that different. But if you want it you can get the same 128GB/sec from socket AM5 as the RX 6400 right now. Which is the class of GPU they could be integrating, instead of the existing ones which are <17% as fast. It's relatively uncommon for the CPU and GPU to be memory bound on the same workload. Generally it's either one or the other. And there are GPUs, like the RX 6900 XT, that have more CUs per GB/s than the RX 6400. At the same ratio you could have 20 CU with the bandwidth of dual channel DDR5-8000, which will no doubt become more common and less expensive as socket AM5 matures. So the problem isn't the memory bandwidth, it's that the iGPU could be bigger. And nothing stops them from making arbitrarily much memory bandwidth available even in existing sockets by including a few GB of HBM on the CPU package. That's essentially what Apple is doing to the exclusion of even having separate memory, but that's not optimal either. Now you can get a Macbook Pro with 96GB of very fast memory (if you want to pay $4000+), but then it can't run the full ~122GB LLaMA model which anyone can do with ~$200 worth of DDR4 in an ordinary desktop PC. The right answer is to provide a few GB of HBM on the CPU package which can be used as an L4 cache and to provide the bandwidth needed for a fast iGPU, then have memory slots to add an arbitrary amount of ordinary memory so that anything that won't fit there can run at lower speed, instead of not at all. That isn't on offer yet, but it's better, and it's possible, so give it a minute.
- sliken 4y agoThere have been numerous attempts at GPU friendly memory systems for CPUs. Iris Pro had an off chip, but inside package help. I forget if it was cache, ram, or edram. The current apple solution looks like the best I've seen (100,200,400, or 800GB/sec). Frustratingly the PS5 and XboxX both had pretty nice memory systems for CPU+IGPU, but during the entire GPU shortage AMD never brought it to market for PCs/ laptops. Although there is hope. The AMD Strix Halo, due in 2nd half 2024 does claim to have a 256 bit wide memory bus. Sure a fancy hybrid that uses a few fast stacks of HBM and some slower dimm slots is possible, but it's far from sure if such a design would hit the size, power, performance, and cost constraints of the market. Clearly the Apple 400GB/sec apple solution fits in a thin/light laptop with pretty good battery life.