4 ms·
The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
by jwr 10mo ago
The author seems unaware of how well recent Apple laptops run LLMs. This is puzzling and puts into question the validity of anything in this article.
- fancyfredbot 10mo agoI think the author is aware of Apple silicon. The article mentions the fact Apple has unified memory and that this is advantageous for running LLMs.
- dangus 10mo agoThen idk why they say that most laptops are bad at running LLMs, Apple has a huge marketshare in the laptop market and even their cheapest laptops are capable in that realm. And their PC competitors are more likely to be generously specced out in terms of included memory. > However, for the average laptop that’s over a year old, the number of useful AI models you can run locally on your PC is close to zero. This straight up isn’t true.
- DANmode 10mo ago> Apple has a huge marketshare in the laptop market Hello, from outside of California!
- dangus 10mo agoGlobal Mac marketshare is actually higher than the US: https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-the-u-s-pc-market-and-17-1-of-the-global-pc-market/ https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...
- layer8 10mo agoBy “PC”, they mean non-Apple devices. Also, macOS only has around 10% desktop market share globally.
- dangus 10mo agoIt's actually closer to 20% globally. Apple now outsells Lenovo: https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-the-u-s-pc-market-and-17-1-of-the-global-pc-market/ https://www.mactech.com/2025/03/18/the-mac-now-has-14-8-of-t...
- layer8 10mo agoI meant market share in terms of installed base: https://gs.statcounter.com/os-market-share/desktop/worldwide/ https://gs.statcounter.com/os-market-share/desktop/worldwide...
- dangus 10mo agomacOS and OS X are split on this graph, and “Unknown” could be anything? This might actually show Apple install base close to 20%.
- andai 10mo agoSo I'm hearing a lot of people running LLMs on Apple hardware. But is there actually anything useful you can run? Does it run at a usable speed? And is it worth the cost? Because the last time I checked the answer to all three questions appeared to be no. Though maybe it depends on what you're doing? (Although if you're doing something simple like embeddings, then you don't need the Apple hardware in the first place.)
- DANmode 10mo agoOf course it depends what you’re doing. Do you work offline often? Essential.
- jki275 10mo agoI can definitely write code with a local model like Devstral small or a quantized granite, or a quantized deep-seek on an M1 Max w/ 64gb of ram.
- fhsm 10mo agoThis paper shows a use case running on Apple silicon that’s theoretically valuable: https://pmc.ncbi.nlm.nih.gov/articles/PMC12067846/ https://pmc.ncbi.nlm.nih.gov/articles/PMC12067846/ Who cares if result is right / wrong etc as it will all be different in a year … just interesting to see a test of desktop class hardware go ok.
- seanmcdirmid 10mo agoI have an MBP Max M3 with 64GB of RAM, and I can run a lot at useful speed (LLMs run fine, diffusion image models run OK although not as fast as they would on a 3090). My laptop isn't typical though, it isn't a standard MBP with a normal or pro processor.
- sueders101 10mo agoI've tried out gpt-oss:20b on a MacBook Air (via Ollama) with 24GB of RAM. In my experience it's output is comparable to what you'd get out of older models and the openAI benchmarks seem accurate https://openai.com/index/introducing-gpt-oss/ https://openai.com/index/introducing-gpt-oss/ . Definitely a usable speed. Not instant, but ~5 tokens per second of output if I had to guess.
- fancyfredbot 10mo agoMost laptops have 16GB of RAM or less. A little more than a year ago I think the base model Mac laptop had 8GB of RAM which really isn't fantastic for running LLMs.
- literalAardvark 10mo agoApple has a 10-18% market share for laptops. That's significant but it certainly isn't "most". Most laptops can run at best a 7-14b model, even if you buy one with a high spec graphics chip. These are not useful models unless you're writing spam. Most desktops have a decent amount of system memory but that can't be used for running LLMs at a useful speed, especially since the stuff you could run in 32-64GB RAM would need lots of interaction and hand holding. And that's for the easy part, inference. Training is much more expensive.
- nunodonato 10mo agomy laptop is 4 years old. I only have 6Gb VRam. I run, mostly, 4b and 8b models. They are extremely useful in a variety of situations. Just because you can't replicate what you do in chatgpt doesn't mean they don't have their use cases. It seems to me you know very little about what these models can do. Not to speak of trained models for specific use cases, or even smaller models like functiongemma or TTS/ASR models. (btw, I've trained models using my 6Gb VRAM too)
- literalAardvark 10mo agoA few examples of useful tasks would be appreciated. I do suffer from a sad lack of imagination.
- nunodonato 10mo agoI suggest taking a look at /r/localLLaMa and see all sorts of cool things people do with small models.
- reactordev 10mo agoI’ll chime in and say I run LM Studio on my 2021 MacBook Pro M1 with no issues. I have 16GB ram. I use unsloth quantized models like qwen3 and gpt-oss. I have some MCP servers like Context7 and Fetch that make sure the models have up to date information. I use continue.dev in VSCode or OpenCode Agent with LM Studio and write C++ code against Vulkan. It’s more than capable. Is it fast? Not necessarily. Does it get stuck? Sometimes. Does it keep getting better? With every model release on huggingface. Total monthly cost: $0
- dangus 10mo agoYeah, any Mac system specced with a decent amount of RAM since the M1 will run LLMs locally very well. And that’s exactly how the built-in Apple Intelligence service works: when enabled, it downloads a smallish local model. Since all Macs since the M1 have very fast memory available to the integrated GPU, they’re very good at AI. The article kinda sucks at explaining how NPUs aren’t really even needed, they just have potential to make things more efficient in the future rather than depending on the power consumption involved with running your GPU.
- whazor 10mo agoBut economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI. However, I agree with the article that people will run big LLMs on their laptop N years down the line. Especially if hardware outgrows best-in-class LLM model requirements. If a phone could run a 512GB LLM model fast, you would want it.
- ignoramous 10mo ago> economically, it is still much better to buy a lower spec't laptop and to pay a monthly subscription for AI Uber is economical, too; but folks prefer to own cars, sometimes multiple. And how there's market for all kinds of vanity cars, fast sportscars, expensive supercars... I imagine PCs & Laptops will have such a market, too: In probably less than a decade, may be a £20k laptop running a 671b+ LLM locally will be the norm among pros.
- joshred 10mo agoPaying $30-$70/day to commute is economical?
- ignoramous 10mo ago> Paying $30-$70/day to commute is economical? When LLM use approaches this number, running one locally would be, yes. What you and other commentator seem to miss is, "Uber" is a stand-in for Cloud-based LLMs: Someone else builds and owns those servers, runs the LLMs, pays the electricity bills... while its users find it "economical" to rent it. (btw, taxis are considered economical in parts of the world where owning cars is a luxury)
- zmmmmm 10mo agoif you calculate depreciation and running costs on a new car in most places - I think it probably would be.
- adrianN 10mo ago
- azuanrb 10mo agoYou still need ridiculously high spec hardware, and at Apple’s prices, that isn’t cheap. Even if you can afford it (most won't), the local models you can run are still limited and they still underperform. It’s much cheaper to pay for a cloud solution and get significantly better result. In my opinion, the article is right. We need a better way to run LLMs locally.
- almosthere 10mo ago749 for an M4 air at Amazon right now
- tossandthrow 10mo agoTry running anything interesting on these 8gb of ram. You need 96gb or 128gb to do non trivial things. That is not yet 749 usd
- jki275 10mo ago64gb is fine.
- kibwen 10mo agoThis subthread is about the Macbook Air, which tops out at 32 GB, and can't be upgraded further. While browsing the Apple website, it looks like the cheapest Macbook with 64 GB of RAM is the Macbook Pro M4 Max with 40-core GPU, which starts at $3,899, a.k.a. more than five times more expensive than the price quoted above.
- jki275 10mo agoI have an M1 Max w/ 64gb that cost me much less than that -- you don't have to buy the latest model brand new.
- seanmcdirmid 10mo agoif you are going for 64GB, you need at least a Max CPU or you will be bandwidth/GPU limited.
- cmxch 10mo agoOnly if you want to take all the proprietary baggage and telemetry that comes with Apple platforms by default. A Lenovo T15g with a 16gb 3080 mobile doesn’t do too badly and will run more than just Windows.
- pimeys 10mo agoI just got a Framework desktop with 128 GB of shared RAM just before the memory prices rocketed, and I can comfortably run many even bigger oss models locally. You can dedicate 112GB to the GPU and it runs Linux perfectly.
- selinkocalar 10mo agoThe M-series chips really changed the game here
- reactordev 10mo agoThis article is to sell more laptops.
- gcanyon 10mo agoIf Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price. And as far as I understand it, an Air with an M3 is perfectly capable of running larger models (albeit slower) if it had the memory.
- fnord77 10mo agoit's astonishing how apple gouges on the memory and ssd upgrade prices (I'm on an M1 w/ 64Gb/4Tb). That said they have some elasticity when it comes to the DRAM shortage.
- aurareturn 10mo agoThey gouge you on RAM and SSD but provide a far better overall machine for the price than Windows laptops.
- Eggpants 10mo agoThe M-series unified memory is built into the chip itself, not separate components. Of course Apple is going to maintain their margins, but it’s easy to see why with this design more memory is more expensive than drams. Well maybe not with the current market pricing which hopefully is temporary.
- mft_ 10mo agoYou’re not wrong that Apple’s memory prices are unpleasant, but also consider the competition - in this context (running LLMs locally) laptops with large amounts of fast memory that can be purposed for the GPU. This limits you to Apple or one specific AMD processor at present. An HP Zbook with an AMD 395+ and 128Gb of memory apparently lists for $4049 [0] An ASUS ROG Flow z13 with the same spec sells for $2799 [1] - so cheaper than Apple, but still a high price for a laptop. [0] https://hothardware.com/reviews/hp-zbook-ultra-g1a-128gb-review https://hothardware.com/reviews/hp-zbook-ultra-g1a-128gb-rev... [1] https://www.hidevolution.com/asus-rog-flow-z13-gz302ea-xs99-13-4-wqxga-180hz-touch-ryzen-ai-max-395-128gb-ram.html https://www.hidevolution.com/asus-rog-flow-z13-gz302ea-xs99-...
- terafo 10mo agoThis article specifically talks about PC laptops and discusses changes in them.