6 ms·
Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market. That m
by MisterKent 3mo ago
Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market.
That means they're actually incentivized at least short term, to benefit PCs becoming strong enough to do local LLMs. Which makes this play make even more sense. Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
- matheusmoreira 3mo agoIndeed. If Apple makes it feasible to run models like GLM 5.2 at home, I will become their customer.
- jeron 3mo agoyou're not a customer of any of their products at all already? not a single apple device in your household?
- matheusmoreira 3mo agoCorrect. Been using Linux and Android for over ten years. My household had no Apple devices until I got married.
- AgentElement 3mo agoSome folks like to have a computing environment free of proprietary influences and extremely strong vendor lock-in. I cannot claim to posses any apple devices.
- rjrjrjrj 3mo agoHow does the Mac have extremely strong vendor lock-in? Sure, you can use the App Store and use all the stuff that integrates with iPhone, iCloud, etc But you can also just treat it as Linux for Laptops (that actually works), and roll with all the standard open source tools.
- seabrookmx 3mo agoI don't disagree with you, but technically speaking MacOS is still proprietary and Asahi is not compatible with the latest and greatest Apple devices. While they don't _prevent_ Asahi from doing what they're doing, they certainly don't go out of their way to make it easy for them.
- rjrjrjrj 3mo agoI wasn't thinking of Asahi. Just pointing out that you can run all the standard unix/open source tools and apps on Mac OS (vi, git, qgis, blender, vsc, python, node, etc). With the advantage of higher quality hardware and generally less fiddling. But if you don't like it, switch. I don't see vendor lock-in.
- rhdjcnfj373 3mo agoA lot of convenient things come with the lock-in if you have >1 device Notes sync, Copy/Paste would be hard to give up and took zero effort
- nicoty 3mo agoI don't now how the MacOS equivalent compares, but Linux/Windows has KDE Connect that I use for that https://kdeconnect.kde.org/ https://kdeconnect.kde.org/
- croes 3mo agoApple is also iOS
- NamlchakKhandro 3mo agoNot. A. Single. One.
- esperent 3mo agoNo, I've never owned an Apple device in my life, neither has anyone in my family to my knowledge.
- seabrookmx 3mo agoI didn't have a single Apple device in my house until a month ago when I bought a Neo. The last Apple devices I had before that were an iPod Nano and a PowerMac G5 many many years ago. Apple has pretty good competition in every segment with the exception of maybe the iPad, but I'm not a tablet user.
- fhn 3mo agothere a many people who don't own Apple. Why are you so surprised? I certainly don't and never will. What's it got that I can't get on a standard PC + Linux?
- kombine 3mo agoNone. And I have a PC, a personal laptop, a work laptop, my current and my previous Android phone.
- nchmy 3mo agoWhat a bizarre bubble you live in to even be asking this question... I've never owned a single apple product, and never will. And in the rare occasions in which I have to use someone's MacBook, I'm completely lost - like some elderly person.
- Abishek_Muthian 3mo agoIt's plausible but is the Apple Tax for a 1TB memory machine on top of current memory prices really worth it? I paid around $4000 for 4090m laptop with 16GB VRAM back in 2023, it's great but DoA for even quantized LLMs. I can run SLMs and fine tune it but that's it. We need one of those specialized inference chip startups to succeed and a PC manufacturer willing to bet on them against Nvidia for the local AI to find mass market appeal.
- lowkeyokay 3mo agoI recently bought a Mac mini M4 16 GB - mostly to run Immich. I assumed I needed a Linux box. After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax.
- ramesh31 3mo ago>" After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax." Apple has always been the most cost effective choice for the value you get going all the way back to the Apple II, it's just that the floor of that cost has always been high. Anyone who thinks otherwise is a just a fanboy one way or the other.
- Abishek_Muthian 3mo agoThat's true only for the entry level macs. My M4 Mac Mini has the best Performance/value. But my workstation laptop with 32 cores, 96GB DDR5, Nvidia GPU costs lesser than Macs with lesser performance; not to mention I upgraded the RAM post purchase.
- overfeed 3mo ago> Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs. "Death knell" is a touch hyperbolic. Hardware that can only run quantized models that take up GBs in VRAM falls short of even an A100 (by almost an order of magnitude[0]), which in turn falls short of what an 8xH100 cluster can do (also by another order of magnitude[0]). I'm an avid believer in local LLMs, but I cannot deceive myself - data center accelerators will win on power dissipation numbers alone[1], even when giving generous allowances for higher efficiency on Apple chips - and assuming the Apple-efficiency advantage persists on the same TSMC process node. 0. Based on my unscientific fine-tuning training experiments across local and rented GPUs. YMMV for inference. 1. Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
- kennywinker 3mo agoDoesn’t need to be a winner head to head. If it can do 90% of the tasks the big boys do, at 50% speed, for virtually no extra overhead cost save for the power consumed by a prompt - that’s gonna work for a lot of people. And that’s also basically where we’re at today. Qwen3.6 35b running quantized on 10 year old hardware solves basically all of my uses cases for agents except for coding. The frontier models are faster, and better at coding, but not so much that i’ll pay $200/month for them.
- drnick1 3mo agoConsider this. One of the smallest Qwen models (4B parameters) powers my home automation voice assistant, and runs on CPU alone at >20 tok/s. It is enough for that use case, and could be made even better/faster with a modest GPU. It isn't as smart as some cloud-connected thingamajig, but I would never allow a literal Google or Amazon bug in my home. Huge SOTA models aren't relevant everywhere. Most people use LLMs for rather trivial tasks such as finding typos or drafting text.
- dainiusse 3mo ago
- VeejayRampay 3mo agoI really wish people stopped saying things "I've been saying that" why not just say "I think that" do you see yourself as some kind of visionary about this particular topic? literally EVERYONE is saying that, it's the most obvious fact about AI
- rajnathani 3mo agoTangential: About 8 years ago ex-Apple chip engineers left to design server-grade chips, this was Nuvia, and they got sued by Apple to the point that they had to get acquired by Qualcomm.
- kasabali 3mo agoThen, after getting acquired by Qualcomm they got sued by ARM. So maybe they were assholes.
- seabrookmx 3mo agoOr maybe corporations hate competition. We'll never know!
- genxy 3mo agoI worked at a hyperscaler when the M1 came out. A MacBook Air M1, running a Linux VM was faster and more energy efficient than anything we had in the data center.
- ac29 3mo agoI'm certainly willing to believe the M1 was the most energy efficient, but your data center didnt have anything faster than a laptop?
- genxy 3mo agoNope, across all benchmarks, Linux running in a VM on the M1 had higher perf (4 cores) than any instance type I launched across Intel, AMD and Arm. Triad, coremark, openfoam, verilator, etc. This may have changed, but my hunch is this is still probably true with M5 vs all the cloud providers.
- Marsymars 3mo agoThat's not especially surprising for low thread counts. e.g. if you look at the current Geekbench single core charts, the fastest device across Android/iOS/Mac/PC is the M5 MacBook Pro. Second-fastest is the M5 iPad Pro. The fastest PC CPU (9950X3D) is about 80% the performance of the M5. (And the 9950X3D is about on par with the A18 in the iPhone 16 from 2024.) (All the usual caveats about geekbench scores apply, but they're not nothing.)
- 6thbit 3mo agoThey do stand in front of a great opportunity that would also benefit consumers, which seems rare in the llm era. If people can get opus4.6/gpt5.5-like models locally, labs could raise their prices and sell token speed, better reasoning, mobile-focused improvements, you name it. Not all consumers are power users and many will be happy to pay for flexibility.
- interludead 3mo agoMost people don't actually want to manage models, updates, context limits, quantization, etc. They just want the thing to work everywhere
- rbanffy 3mo agoOnce one person figures that out and writes a blog post, everybody else can do it.
- oblio 3mo agoYes, just like 90% of regular users set up NASes instead of just using Dropbox or Google Drive. https://xkcd.com/2501/ https://xkcd.com/2501/
- rbanffy 3mo agoThe comparison is against your own computer, which is powerful enough, no need to add a box and set it up, and a very expensive cloud service. It’s nothing like setting up a NAS versus using iCloud (because we are talking Apple here).
- interludead 3mo agoI'm not sure it's a death knell for frontier labs so much as a narrowing of what people need them for
- oblio 3mo agoWhen you've raised hundreds of billions in funding, every result except "to the moon" is a death knell.
- oblio 3mo agoLOL at the downvotes. I'm sure that the shareholders aka wealthy people at the top have a lot of patience for huge gambles that don't pay off. If OpenAI and Anthropic turn out to be the IT equivalent of mom and pop stores 5 years from now, their current shareholders will be ecstatic!
- ksec 3mo ago>AND basically no play I'm the hyperscalar market. I do wish they have Xserve back, or a Mac Pro that is Rack Based and support multiple node with M6 Ultra. The Hyperscaler market is so large along with AI their old business case of Xserve didn't make sense no long hold true.