AMD/Intel CPUs for desktops/laptops are mostly 128 bit wide. So these days on laptops (no DIMM slots) is DDR5-6400 = 102GB/sec peak to DDR5-8000 (128GB/sec pea

AMD/Intel CPUs for desktops/laptops are mostly 128 bit wide. So these days on laptops (no DIMM slots) is DDR5-6400 = 102GB/sec peak to DDR5-8000 (128GB/sec peak). Apple however has 192 bit (m3 pro), 256 bit (m2 pro), 512 bit (m3 max), and 1024 bit (m2 ultra) wide for improved iGPU performance. Generally CPU unified memory systems spend more transistors on cache than GPUs, so you also get some advantages there. iGPUs also have other advantages like much higher max ram, lower latency from CPU -> video memory than the traditional CPU -> PCIe -> GPU -> vram. Depending on what you are sending the cache coherent nature of iGPUs means you can have a complicated data structure and hand it to the GPU with a pointer instead of the normal CPU memory -> serialize -> transfer over PCIe -> poll or tell the GPU it's there, etc. So sure a gGPU is faster at LLMs, but unless you spend $$$$ on GPUs you can get much more ram with an iGPU. So even llama 70B runs decent on a M3 max (prompt @ 19 tokens/sec and eval rate at 8.5 tokens/sec). So sure it's not as fast as a dGPU, but I'd rather run 70B at 8-19 tokens/sec then not be able to run it on a 4090. The widely rumored AMD Strix Halo will have a 256 bit wide memory interface, which should significantly help the iGPU, much like the Apple M2 Pro.