12 ms·

Apple Silicon Exec Explains Mac Mini AI Demand and On-Device Future

• sublinear 4mo ago
  > "The speed of AI development right now is just crazy," Brooks said. "I can't imagine where we're going to be a year from now, three months from now, or even a month from now," he added. I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI. Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search. Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.
  • paul7986 4mo ago
    The new Siri isnt that exciting at least on my iPhone 15 Pro Max. I know it's beta but it's sluggish and often says try again. I watched many videos on Youtube saying its amazing but maybe not so much on older phones? Also, I need a Siri I can talk in an unfettered manner from my lock screen while Im driving without having to unlock the screen. Probably a big ask for Siri to know my voice via voice fingerprint allowing unfettered access from my lock screen. Others running the beta now on newer iPhones and enjoying it more so?
    • etchalon 4mo ago
      The best models for new Siri, on device, requires the iPhone 17 Pro, so your experience on a 15 is going to be degraded.
    • y1n0 4mo ago
      I use it on my 15 pro max. It's a major improvement, but I still don't use siri much. It's gone from murderously infuriating to passably usable. I still use gpt for any sort of exploratory conversations. I haven't bothered trying to use siri for that because to me that's not what siri is for. Although given how effed up the voice for chatgpt is now with the latest updates I might talk with siri more. Because I use carplay in tandem with my phone where the map is on the carplay screen and turn by turn directions are on my phone, it's always unlocked so I haven't run into whatever lock screen issue you brought up.
    • helsinkiandrew 4mo ago
      > The new Siri isnt that exciting at least on my iPhone 15 Pro Max On my Pro 16 it has its ups and downs - I still can't get it to "play my running playlist on shuffle" whilst running (this is the only thing I used Siri for before the beta and it would improve my life immeasurably if it worked). But it responds to things like "how long will it take to drive to the AirBnb booking in my inbox", and "when is X playing a concert in Y - add a calendar entry with details" perfectly. This is a beta and I have hopes, but I can imagine it will run better on a 17 and later
      • mark_l_watson 4mo ago
        Those things also work for me but Siri is often so slow that it would be faster to do myself.
  • ZaoLahma 4mo ago
    > Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search. This is... a view. Maybe I live in a strange sphere of strange ("normie"-ish) people, but the people around me are for sure using AI. Mostly chatgpt to be fair. They use it to compare products that they intend to buy, identify plants in nature, create travel plans, find interesting places to visit nearby, give movie suggestions based on what they have previously enjoyed and so on and so forth. AI is becoming a very integrated part of their reality. To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse.
    • com2kid 4mo ago
      Chatgpt is so damn good for cooking it is unbelievable. It will learn your family's tastes over time, you can tell it what pantry staples you keep is stock, and you can take any recipe you find online and ask for stuff like "find a way to make this preparable in 45 minutes instead of 2 hours, what trade offs will I be making?"
      • sandspar 4mo ago
        I agree! I think it's because there's so much cooking material in its training data. I wonder what proportion of the internet is food blogs and recipes... Probably a lot!
    • PhilipRoman 4mo ago
      This is true but I can do the same on free ChatGPT without even logging in. I wouldn't pay $5 a month for that functionality, much less $20 (or >$1000 to be able to run it at home). SOTA AI for "serious" work is in a different position, used by fewer people but with big pockets and sometimes a pathological dependence on it.
    • bcjdjsndon 4mo ago
      > To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse. I can see this kind of low level usage as being perfect for local LLMs... So I can't see a market there for openai etc forever. Are they using the free chatgpt or a paid one?
  • camillomiller 4mo ago
    Nobody does what to do, they are just throwing trillions at it betting they’ll figure out. If they don’t they’ll be screwed, if they do Apple will quickly catch up with a better and more refined product, as they’ve always done.
  • bcjdjsndon 4mo ago
    If you take just this part of the quote... >"I can't imagine where we're going to be a year from now, three months from now, or even a month from now," I'd say he's making an accurate appraisal of his abilities
• onion2k 4mo ago
  Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind. Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.
  • kamranjon 4mo ago
    I dunno I use LMStudio pretty regularly and the MLX folks and the community usually have MLX versions of new model releases up within a day or two.
    • onion2k 4mo ago
      For some models like the popular coding and chat models, things move faster. For things like images, voice, sound etc they definitely lag a long way behind.
    • Barbing 4mo ago
      & they clearly delineate all the models that’ll work on your exact machine (but guardrails can be disabled in settings)
  • throw1234567891 4mo ago
    Converting a model to mlx is literally a git clone and mlx_lm.convert. 5 minute operation after clone.
    • fellowmartian 4mo ago
      Or just ask any model to do it for you.
  • gizajob 4mo ago
    How is it a pain exactly? It’s just learning and only takes a day or two to get up to speed. We seem to have forgotten that for the past fifty years doing all kinds of tasks on computers has been tedious and involved and time consuming to even get working. My first computer had 48kb of RAM and to play a game you had to load it off cassette for five minutes. That was annoying. Having LM Studio download a model and load so you can chat or attach an agent it is effortless and easy in comparison.
• majestik 4mo ago
  Ok, I didn’t want to take the bait but this one’s just too much. > “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.” Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!” <neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac! > “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.” <neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone! How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens. Was that so hard?
  • Aperocky 4mo ago
    But why should apple invest in developing their own models? Why would it be correct? Or phrase it in a very similar ask, why don't they invest in power plants? The model space is truly crowded, what do they gain or recover suppose they are SOTA? Across the Pacific they are pumping out free models that are only 6-12 months behind. What business sense does it make for Apple to develop their own models?
    • mathisfun123 4mo ago
      apple has invested in training models https://machinelearning.apple.com/research/introducing-third-generation-of-apple-foundation-models https://machinelearning.apple.com/research/introducing-third... they just suck
    • mlrtime 4mo ago
      The average apple user doesn't care about models, they care about inference. I agree, apple shouldn't invest in their own models. But they should have close to the best inference + end user design.
  • 4mo ago
• znnajdla 4mo ago
  It’s not for the AI inference, it’s for the tool calls and desktop GUI app workloads and browser. There aren’t any on-device models capable enough of real work that can run on lower end Mac Minis. But for running a few browsers and GUI apps, you’re much better off buying a Mac Mini than paying for a more expensive and worse-performing container in the cloud. Browsers were not designed to run in Linux containers but they run optimally on baremetal desktop OSes. An M4 Mac Mini beats the single core performance of any VM you might rent in the cloud, in terms of raw compute per dollar (Geekbench scores).
  • Den_VR 4mo ago
    At their original price points, a set of four were a great solution for my requirements in tokens/second/$.
• drewda 4mo ago
  This is a paraphrase of https://www.thedeepview.com/articles/how-apple-s-decade-long-bet-on-chips-won-ai https://www.thedeepview.com/articles/how-apple-s-decade-long...
• JumpCrisscross 4mo ago
  Apple could probably sell a machine starting at $10,000 if they architected it as the sole place one’s Private Cloud Compute [1] ran. It would need a path to a $2,500 machine, I think. But this is a niche I don’t think another consumer-facing brand could do like Apple. [1] https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/
  • mmoustafa 4mo ago
    The out-of-stock $6000 M3 Ultra Mac Studios with 800GB+ memory bandwidth are going for $24,000 on eBay, so yes definitely
    • JumpCrisscross 4mo ago
      Let me clarify, I think Apple could sell a device at the scale Apple sells at around the $10 to 25 thousand price point. Like, take out the price sheets for the Apple Car. Then sell me an AI tower at those price points.
      • fy20 4mo ago
        In 2010 one of the standard configurations for the Mac Pro was $4,999. Once you customised ram, storage, peripherals and software it could easily end up above $15,000, or $23k today accounting for inflation. Apple hardware is one thing that has actually got cheaper over time. https://www.macworld.com/article/209019/macguide2010.html https://www.macworld.com/article/209019/macguide2010.html
      • testing22321 4mo ago
        I’m still disappointed they didn’t make a Mac Pro with 4 or 8 or 16 or 32 ultra M chips for something insane
      • bigyabai 4mo ago
        Or they could use that same amount of memory to ship 64x Macbook Neos, and probably make higher margins off the hardware volume. Those Macbook Neo users would be very reliant on Apple intelligence, enough maybe to pay for a service with it. I think Apple's much happier going this path.
• huragok 4mo ago
  If I had the capital I’d make an household inference appliance. No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility. Even the appliance faceplate would be a display showing stats like an old HiFi stereo. Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory
  • Aperocky 4mo ago
    You're describing the mac mini/studio with some facelift.
    • boredatoms 4mo ago
      Yeah but like running linux hopefully
      • throw1234567891 4mo ago
        so you have invent unified memory for linux first because that’s the limitation today
        • Alpha3031 4mo ago
          Fairly sure most iGPUs these days are zero-copy and can dynamically allocate memory so what does "unified memory" mean to you exactly? A wider bus would be nice but it's not exactly a groundbreaking new invention.
          • throw1234567891 4mo ago
            I was actually pretty far off: > Unified memory in Linux creates a single address space accessible to both the CPU and GPU, eliminating the need to manually copy data between system RAM and video memory. It is enabled via NVIDIA's CUDA, AMD's ROCm/HIP, or generic kernel-level Heterogeneous Memory Management (HMM). So it does exist and is available for platforms that matter.
• iknowstuff 4mo ago
  Oh please the neural engine is mostly useless for LLMs. Siri in iOS 27 is laughably pathetic and slow compared to GPT Live DESPITE sending personal context to their (attested) cloud to execute anything but the most basic queries. Still years behind.
  • apparent 4mo ago
    I think the relevant comparison is the developer beta, which has access to the Gemini-powered Siri that will roll out publicly later this year. From the reviews I've seen, Apple won't be "years behind" (which surely they were) for long.
    • SietrixDev 4mo ago
      But not in the EU, because of the DMA :/
    • mark_l_watson 4mo ago
      I am on the developer beta for iOS/iPadOS/macOS and all I see is still just the OpenAI option. That said, Siri seems a little bit better now - my subjective opinion. It is a little bit less frustrating.
    • iknowstuff 4mo ago
      I did say iOS 27. I am using the new Siri. It’s better, but extremely unreliable (half of all requests fail) and slow. It should be like a Codex running on my phone with the ability to chain skills (intents) to execute a task, but it’s too crappy for that. And the voice is still a poor text to speech model, very far behind GPT live.
• chvid 4mo ago
  I don’t understand why they are not reintroducing the Apple Xserve?
  • ben_w 4mo ago
    My guess is the OS. People who want a server often enough want to choose the OS, Apple wants to supply the OS and the hardware together so they're not blamed every time the two turn out to be incompatible, as happened the other way round in the 90s.
  • littlecranky67 4mo ago
    Target audience - B2B. There are multiple videos of Steve Jobs saying that he hates B2B, because the people using the devices are not the ones making the purchasing decision. It is pretty much against Apples DNA, and all their B2B they have today is a means for them to sell more B2C.
  • pjmlp 4mo ago
    Because they killed the market, no one would now buy a macOS server, when Linux distributions, and to a lesser extent FreeBSD, own the server room. They would even sell less than Windows Server licenses. By the way, they are down the same path with the workstation market, now that they only top level answer is the Mac Studio. Workstation market wants flexible towers that they can customise to their own liking and special use cases. The main reason Swift exists for Linux, is that app developers need to have servers somewhere, and if they want to share Swift code with the backend, well it isn't going to be on macOS Server.
    • not-a-llm 4mo ago
      the SemiAnalysis guy argues that there is a market for macOS servers - running macOS VMs that agents need for computer use. some software is mac only but Apple needs to change the licensing model, currently you are allowed to run only 2 macOS VMs for every physical one you buy
    • mrbnprck 4mo ago
      At the same time Apple made remotely hosting a Mac much easier.
• pantulis 4mo ago
  Being a Mac switcher since 2003 I am as much of a fanboy as anybody else but this quote from the article caught my attention, and smells like PR. > Many AI tools are also Mac-first or Mac-only I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?
  • ikari_pl 4mo ago
    one. apfel.
  • esskay 4mo ago
    oMLX[1] is the only one that comes to mind but it's not exactly unique to mac, it just runs MLX models and provides a nice gui. It does have the whole paged SSD KV cache thing, not sure if thats working on other platforms. [1] https://omlx.ai/ https://omlx.ai/
• fsuts 4mo ago
  Apple doesn’t have an ads business, and it is inevitable that Google/openai/Anthropic are going to seek to monetise consumer ai via ads. So the ad free Apple on device experience will be welcome.
  • Citizen_Lame 4mo ago
    What does this mean? Apple ads bring in 8 billion revenue a year.
    • fsuts 4mo ago
      They aren’t as aggressive and focus on the customer experience more than others, 8bn isn’t a lot for a company with their reach. For example Apple radio is a free product with no ads, Apple TV and Apple Music don’t have an ads supported tier.
      • amelius 4mo ago
        You think Apple is a saint, but they have a completely locked down mobile computing/phone platform. Why the trust in them doing what is best for the user?
        • fsuts 4mo ago
          I didn’t say Apple are a saint I said as they don’t push ads as much as others, the customer may be better As ads focused services are designed to keep you there as long as possible rather than delivering what you are actually want quickly
          • amelius 4mo ago
            It's still wishful thinking. A better strategy is to not root for one player and vote (with your wallet) for more competition.
    • mlrtime 4mo ago
      Advertising accounts for about 1% to 2% of Apple's total overall revenue, bringing in roughly $8.5 to $12 billion annually. 1-2% is rounding error.
• pjmlp 4mo ago
  > Apple's Mac mini and Mac Studio have become the machines of choice for running AI agents, according to Doug Brooks, Apple's senior product manager of Apple silicon. This is mostly an US phenomenon, no Mac mini nor Mac Studio around here. Only Thinkpads and Macbooks laptops talking to hyperscalers.
  • esskay 4mo ago
    Around where? They're pretty popular for it in the UK right now given our obscene energy pricing as they end up being one of the best low power options for local llm. If you're not in the local llm space you obviously wouldn't see it. It's like saying Tennis isnt popular around here then admitting you dont frequent a tennis court so wouldn't even know.
    • pjmlp 4mo ago
      Around here in my German office, and down in my home country Portugal, even less, given the average salaries.
    • walthamstow 4mo ago
      So, when you said pretty popular, you meant in the local LLM space, which is a niche of a niche.
• khurs 4mo ago
  People bought diesel cars as miles per gallon was higher than petrol. People are buying apple unified as electricity costs in many countries are very high, so cheaper to run than Nvidia setup. As non-apple unified memory options increase, many people will have more choose those
  • catlikesshrimp 4mo ago
    [dead]
• Sebastian_09 4mo ago
  What I’m not sure to understand is that if you want to just run Claude code or openclaw type software with llm apis or subscriptions (and not run local models) to benefit from a local file system and always-on capability for ‘second brain’ type of workflows, I guess you don’t need a Mac mini but can run it on a raspberry pi or an old laptop ? Does anyone have experience with that ?
  • windex 4mo ago
    Yup, for openclaw and APIs you dont need a big PC. I run something lightweight on the RPI4 8gb. Many people run local LLMs which is where a mac is useful. Frankly I dont think you can beat the value of an openrouter subscription and API calls.
    • embedding-shape 4mo ago
      > Many people run local LLMs which is where a mac is useful Unless you go for the very expensive options, most of the Mac Minis really aren't suitable for running local LLMs, they're painfully slow with prefill/processing input, and the models you are able to run don't handle long context very well, which these sort of long-running agents perform very differently with when you can. I'll agree with your latter point, hard to beat the value of using something like OpenRouter or similar remote inference. Even with local models, you can run the agent software and the inference workload on different hosts, which is what I'm doing at home. Beefy server responsible for inference, tiny VM on other server is running the actual agent software + RPC + bridges and what not.
    • SoMomentary 4mo ago
      Why not go direct to the source instead of paying an extra 5.5%? Seems like it'd be trivial to have AI wire up connections to your preferred inference providers and save yourself some money over time.
      • bdcravens 4mo ago
        If you're referring to the markup charged by OpenRouter, you can use harnesses like OpenClaw/Hermes without it and go direct like you're saying. If you're talking about actually "routing", then you don't get that out of the box. However, the popular use of those harnesses doesn't often use the smart routing approach with a single agent. Instead, the approach is to create multiple agents, each with a role and a model tailored to that role by cost and functionality.
• embedding-shape 4mo ago
  > "people often want a system that's under their control, isolated from their primary machine, and capable of running 24 hours a day, seven days a week," said Brooks. "A Mac mini is an amazing system for that," he added. These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing. Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.
  • groundzeros2015 4mo ago
    I think tech people spend hundreds on tech because it’s fun.
  • tomaskafka 4mo ago
    The remote access story for macOS is absolute sadness, without Jump Desktop there would be zero performant ways to access that “system under my control”. And all of that because Tim Apple fears any feature that could mean people could have less than one iDevice per person.
    • lucky_cloud 4mo ago
      I'm curious what you mean. I have been accessing Macs remotely over SSH and VNC for like 20 years and it's always been easy and as performant as the network would allow.
      • cosmic_cheese 4mo ago
        The built in high rez screen sharing between Macs works well too. Through Tailscale I’ve accessed my main Mac from both the opposite coast of the US and from the other side of the Pacific and it works great. Host resolution automatically matches that of client, image quality is great, framerate is decent, latency is minimal. The host creates virtual screens for the connection so connected screens don’t light up and the machine remains locked to anybody accessing it physically too, which is a nice privacy assurance.
• theplumber 4mo ago
  The issue with the Apple is that they didn’t really develop any competitive local AI machine. Their strategy/marketing falls flat when you ask them how exactly they implement AI: they buy it from Google cloud. In the future local AI may become a thing but that’s 4-5 years away. I count the 2q-4q and atrocious performance as “local ai” only for the enthusiasts crowd not for people doing competitive work.
• api 4mo ago
  Apple could dominate this niche if they decided, for a while until prices fall, to eat some margin and bump up RAM in high end models. Couple that with a new M series chip with even faster AI performance. It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists. For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.
• jshier 4mo ago
  Now just imagine they'd kept making the Mac Pro and enabled compute offload to GPUs. Or even just passthrough to Linux VMs. Would've been quite the AI machine.
• pstoll 4mo ago
  Except that Mac ultra M3 they talk about is now only being sold in the 96 GB configuration. It’s no longer being sold in larger Ram configurations by Apple Apple because of the global RAM shortage. And you can not add memory after purchase because it’s integrated/soldered on.
  • zer00eyz 4mo ago
    > And you can not add memory after purchase because it’s integrated/soldered on. For as much as I dont like this aspect of modern computing, I understand why it is done from a technical perspective. Power, heat, and performance are all "better" when ram is on the motherboard vs in a "stick".
• habosa 4mo ago
  Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat. Just writing this down so I can be praised/mocked in 5 years.
  • giancarlostoro 4mo ago
    Its a shame RAM prices are getting in the way of Apple. The 512 GB RAM Mac Studios would have been worthwhile.
  • QuercusMax 4mo ago
    I'm also of this opinion, but also that it doesn't have to be Apple (but they are well positioned). What I've seen with running local models on my 48GB M4 MBP is really impressive - it's not the same level as hosted stuff, but it's better than what I was using a year or two ago.
  • dabbz 4mo ago
    I suspect we'll see a hybrid before an all or nothing. Local models for computer control or delegating, online models for things that need strong reasoning, planning, and knowledge access. Again, I'd be more than happy to be wrong. I just see models growing faster than the hardware can.
  • xyst 4mo ago
    Apple has failed to live up to the Steve Jobs era and initial iPhone hype. It's not just "AI experiences", it's computing in general. Maybe the consumer sector is just dead/dying. Maybe consumers are just running out of cash because filthy VCs are destroying communities and forcing the 99% into poverty. The one thing that is marginally exciting: the Apple SoC or M series chips. It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
    • overfeed 4mo ago
• kylehotchkiss 4mo ago
  When you don't limit on-device AI to "can be used to run coding model", Mac minis are great. My M4/16gb been working on a long term research project using Qwen 2.5 8b for months now. The performance is good enough for processing a lot of small text prompts.
• devy 4mo ago
  The submission link should have been the original editorial piece at https://www.thedeepview.com/articles/how-apple-s-decade-long-bet-on-chips-won-ai https://www.thedeepview.com/articles/how-apple-s-decade-long...
• enbarca 4mo ago
  On-device isn't only cheaper/faster - for health data it's the whole point! The self-sovereign community desperately needs an affordable AI appliance and Apple is still the most likely candidate to deliver on the privacy promise.