3 ms·

Apple is the only company that makes integrated GPUs that are actually any good, and even their CPUs are really well-optimized for these sorts of workloads. Th

Apple is the only company that makes integrated GPUs that are actually any good, and even their CPUs are really well-optimized for these sorts of workloads. They also build their RAM in a way that makes it suitable for both CPU and GPU uses. This means you don't have separate CPU RAM and GPU VRAM, it's all just memory. If you get more RAM for your Macbook, that automatically means more memory you can use for LLM inference. Yes, you can technically buy Nvidia, but you'd pay just as much for that if not more, and I don't think you can get as much VRAM in a consumer GPU anyway.

IshKebab 2y ago
Haven't laptops used integrated RAM for decades? It was always considered to be a bad thing in the past. I guess because the integrated graphics was always weak?
- hmottestad 2y ago
  One big factor was bandwidth. Seems Apple mostly solved that though.
talldayo 2y ago
> Apple is the only company that makes integrated GPUs that are actually any good I will reiterate; if you have used AMD or Intel iGPUs recently you would know this is just plain wrong. AMD's iGPUs often outperform Apple's for the price, and come with a featureset that isn't intentionally gimped and supports "professional" features like Vulkan conformance. People condemn Apple's GPUs for having a featureset too simple for anything besides video transcoding and compute shaders. If Intel didn't exist, Apple would probably be the single least-efficient and lowest performance GPU designer shipping products today. And they had to cut corners to get there. > If you get more RAM for your Macbook, that automatically means more memory you can use for LLM inference. And then it's your paltry GPU that's the bottleneck. You can get comparable results on pretty much any sub-120b parameter model with PCIe offloading on Nvidia hardware that costs a fraction of a Mac Studio with enough memory to compete. See for yourself. Nvidia's fastest laptop chips still outperform Apple's most powerful desktop GPU chips: https://browser.geekbench.com/opencl-benchmarks https://browser.geekbench.com/opencl-benchmarks And it's not even close. Apple's biggest professional GPUs struggle to swap spit with Nvidia's last-gen laptop chips.

IshKebab 2y ago

Haven't laptops used integrated RAM for decades? It was always considered to be a bad thing in the past. I guess because the integrated graphics was always weak?

hmottestad 2y ago
One big factor was bandwidth. Seems Apple mostly solved that though.

talldayo 2y ago

> Apple is the only company that makes integrated GPUs that are actually any good I will reiterate; if you have used AMD or Intel iGPUs recently you would know this is just plain wrong. AMD's iGPUs often outperform Apple's for the price, and come with a featureset that isn't intentionally gimped and supports "professional" features like Vulkan conformance. People condemn Apple's GPUs for having a featureset too simple for anything besides video transcoding and compute shaders. If Intel didn't exist, Apple would probably be the single least-efficient and lowest performance GPU designer shipping products today. And they had to cut corners to get there. > If you get more RAM for your Macbook, that automatically means more memory you can use for LLM inference. And then it's your paltry GPU that's the bottleneck. You can get comparable results on pretty much any sub-120b parameter model with PCIe offloading on Nvidia hardware that costs a fraction of a Mac Studio with enough memory to compete. See for yourself. Nvidia's fastest laptop chips still outperform Apple's most powerful desktop GPU chips: https://browser.geekbench.com/opencl-benchmarks https://browser.geekbench.com/opencl-benchmarks And it's not even close. Apple's biggest professional GPUs struggle to swap spit with Nvidia's last-gen laptop chips.