3 ms·
Nvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for
by bigyabai 2mo ago
Nvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for compute.
Even the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance than last-gen laptop RTX 4080 chips. And they don't scale, the largest M3 Ultra cluster you can configure is still ~2,000x smaller than a DGX SuperPOD. There's a reason Apple discontinued their rackmount hardware, there's very little demand for Apple Silicon in the datacenter.
- literoldolphin 2mo agoAs I mentioned above you can't take the data center into the cafe somewhere. We're talking about running local models here.
- adrian_b 2mo agoBut you may be able to connect with your laptop to your home server, even from some cafe, and run the LLM remotely. I always connect back home when I am away and I want access to a beefier computer. On the home server, "Wake on LAN" is enabled, so I can power it on and off from my home router.