3 ms·
And yet it's also a laptop you can basically take anywhere unlike a giant video card with 1000 watt power supply requirements.
by literoldolphin 2mo ago
And yet it's also a laptop you can basically take anywhere unlike a giant video card with 1000 watt power supply requirements.
- bigyabai 2mo agoNvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for compute. Even the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance than last-gen laptop RTX 4080 chips. And they don't scale, the largest M3 Ultra cluster you can configure is still ~2,000x smaller than a DGX SuperPOD. There's a reason Apple discontinued their rackmount hardware, there's very little demand for Apple Silicon in the datacenter.
- literoldolphin 2mo agoAs I mentioned above you can't take the data center into the cafe somewhere. We're talking about running local models here.
- adrian_b 2mo agoBut you may be able to connect with your laptop to your home server, even from some cafe, and run the LLM remotely. I always connect back home when I am away and I want access to a beefier computer. On the home server, "Wake on LAN" is enabled, so I can power it on and off from my home router.