3 ms·
Agreed that for GPU oomph, you'd do better with a dGPU; there's some very good deals for dGPU laptops but even this next-tier-down 8050S is still a rather expen
by jauntywundrkind 1y ago
Agreed that for GPU oomph, you'd do better with a dGPU; there's some very good deals for dGPU laptops but even this next-tier-down 8050S is still a rather expensive new purchase by compare (for now; Strix Halo is brand new). But the power consumption will likely be much higher.
Strix Halo as an APU has two very clear advantages. First, I expect power consumption is somewhat better, due to using LPDDR5(x?) and not needing to go over PCIe.
But the real win is that you can get a 64GB or 128GB GPU (well somewhat less than that)! And there's not really anything stopping 192GB or 256GB builds from happening, now that bigger ram sizes are finally available in the world. But so far all Strix Halo offerings are soldered on ram (non user upgradeable, no camm2 offerings yet), and no one's doing more than 128GB. But that's still a huge LLM compared to what consumers could run before! Or lots of LLMs loaded and ready to go! We see similar things with the large unified memory on Mac APUs; it's why are minis are sometimes popular for LLMs.
Meanwhile Nvidia is charging $20k+ for an A100 GPU with 80GB ram. You won't have that level of performance, but you I'll be able to fit an even bigger LLM than it. For 1/10th the price.
There's also a lot of neat applications for DB's or any kind of data-intense processing. Because unified memory means the work can move between CPU and GPU, without having to move the workload. Normally to use a GPU you end up copying data out of main memory then writing it to the GPU* then reading it to do work, and you can skip 2/3rds of these read/write steps here.
There's some very awesome potential for doing query processing on GPU (ex: PG-Strom). Might also be a pretty interesting for a GPU based router, ala PacketShader (2010).
* Note that PCIe p2p-dma / device RDMA / dma-buf has been getting progressively much better, a lot of attention, across the past half deacde, such that say a nic can send network data direct to GPU memory, or a NVMe drive can send data direct to GPU or network, without bouncing through main memory. One recent example of many: https://www.phoronix.com/news/Device-Memory-TCP-TX-Linux-6.16 https://www.phoronix.com/news/Device-Memory-TCP-TX-Linux-6.1...
Giving an APU actually fast ram has some cool use cases. I'm excited to see the lines blur in computing like this, to see
- dragonwriter 1y ago> Meanwhile Nvidia is charging $20k+ for an A100 GPU with 80GB ram. Or, sometime in the next month or so, NVidia GB10-based miniPC form factor devices with 128GB (with high-speed interconnect to allow two to serve as a single 256GB system) from various brands (including direct from Nvidia) for $3000-4000 depending on exact configuration and who is assembling the completed system.