4 ms·

> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework Ahh I was hoping for the opposite: all of

by post-it 4mo ago

> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework Ahh I was hoping for the opposite: all of the existing features of Claude Code but somehow running locally on my laptop's neural engine. A pipe dream on an M2 with 8 GB of RAM, but I had a flicker of hope there.

inickt 4mo ago
Check out this WWDC session. Obviously not going to compete with the frontier models (and I think 8GB is too small anyways), but Apple did demo MLX + OpenCode. https://developer.apple.com/videos/play/wwdc2026/232/ https://developer.apple.com/videos/play/wwdc2026/232/ https://www.youtube.com/watch?v=wykPErJ8M-8 https://www.youtube.com/watch?v=wykPErJ8M-8
FuriouslyAdrift 4mo ago
I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.
- godzillabrennus 4mo ago
  If only we could buy 1TB of unified memory in a Mac for $1k-$2k in total hardware costs. Apple would basically be able to extinguish the entirety of the market cap for Nvidia, OpenAI, Anthropic, and others all at once. In 10 years, I hope my MacBook Pro can run today's frontier models and has 1TB of unified Memory.
  - manoDev 4mo ago
    I’m bullish on Apple because of that. Tech waves always oscillate between mainframe/thin-client models at first, then commodity hardware catches up. Apple is well positioned to deliver that with the M series, all it takes is for the current AI bubble to pop a bit and memory costs go down.
  - dboreham 4mo ago
    The people who train the frontier models want to recover their costs, so they're not going to let you do that.
  - shadowpho 4mo ago
    Why can’t Apple launch a $50k product for $1k? Everyone would buy it!
    - tempoponet 4mo ago
      To go further down this pipe dream - Anthropic / OpenAI would buy them all and still price out the consumer. There's no end-run in this scenario.
      - shadowpho 4mo ago
        Well everyone would. How many would you buy if you could turn around and sell them for $30k easily lol? It’s like saying “well if Subaru launches a nice hybrid suv for $1k it’ll sell like pancakes” and yeah.. but it costs more in steel/ram to build that lol
  - jayd16 4mo ago
    They want you to buy four 256GB Studios and link them with ThunderBolt.
    - Danox 4mo ago
      Yes, particularly if that memory is designed and engineered by Apple in house like Apple Silicon in house and manufactured by TSMC on shore somewhere in the United States.
  - connicpu 4mo ago
    The Nvidia GB300 DGX Station, which isn't even going to hit 1TB total memory, is expected to launch at almost $100k. Bit of a pipe dream with memory prices where they're at.
    - FuriouslyAdrift 4mo ago
      There are multiple server systems available right around the $100k range that have 512B of GPU RAM right now (4x AMD Instinct MI300A) GIGABYTE G383-R80-AAP1 for example
  - bigyabai 4mo ago
    > Apple would basically be able to extinguish the entirety of the market cap for Nvidia I don't think you understand why people buy Nvidia hardware if you're beating the "just add more dual channel DDR, bro" drum. Apple wouldn't even be able to extinguish AMD with a product like that, it's all slow memory being fed into a raster-first GPU architecture.
- pstuart 4mo ago
  The work on LLM in a Flash will probably help, and Apple's NVMe architecture is well suited to maximize throughput could allow their devices to work better on larger models than other vendors.
  - ABS 4mo ago
    [flagged]
satvikpendem 4mo ago
You can use OpenCode or Pi with SSD streaming so it technically will have all the features, just unbearably slow.
jubilanti 4mo ago
> all of the existing features of Claude Code but somehow running locally on my laptop's neural engine You can use environment variables to have claude code query literally any endpoint you choose as long as it has a compatible API.
5701652400 4mo ago
I would not mind if cloud was actually private users iCloud. users pay for it, and it runs in Apple servers next to where users store their iPhotos already. that would be really elegant solution. ..but instead we get Claude, hosted who-knows-where. maybe in X-AI datacenters? maybe in Amazon somewhere? who knows..
- willy_k 4mo ago
  https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/

by post-it 4mo ago

inickt 4mo ago

Check out this WWDC session. Obviously not going to compete with the frontier models (and I think 8GB is too small anyways), but Apple did demo MLX + OpenCode. https://developer.apple.com/videos/play/wwdc2026/232/ https://developer.apple.com/videos/play/wwdc2026/232/ https://www.youtube.com/watch?v=wykPErJ8M-8 https://www.youtube.com/watch?v=wykPErJ8M-8

FuriouslyAdrift 4mo ago

I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.

godzillabrennus 4mo ago
If only we could buy 1TB of unified memory in a Mac for $1k-$2k in total hardware costs. Apple would basically be able to extinguish the entirety of the market cap for Nvidia, OpenAI, Anthropic, and others all at once. In 10 years, I hope my MacBook Pro can run today's frontier models and has 1TB of unified Memory.
- manoDev 4mo ago
  I’m bullish on Apple because of that. Tech waves always oscillate between mainframe/thin-client models at first, then commodity hardware catches up. Apple is well positioned to deliver that with the M series, all it takes is for the current AI bubble to pop a bit and memory costs go down.
- dboreham 4mo ago
  The people who train the frontier models want to recover their costs, so they're not going to let you do that.
- shadowpho 4mo ago
  Why can’t Apple launch a $50k product for $1k? Everyone would buy it!
  - tempoponet 4mo ago
    To go further down this pipe dream - Anthropic / OpenAI would buy them all and still price out the consumer. There's no end-run in this scenario.
    - shadowpho 4mo ago
      Well everyone would. How many would you buy if you could turn around and sell them for $30k easily lol? It’s like saying “well if Subaru launches a nice hybrid suv for $1k it’ll sell like pancakes” and yeah.. but it costs more in steel/ram to build that lol
- jayd16 4mo ago
  They want you to buy four 256GB Studios and link them with ThunderBolt.
  - Danox 4mo ago
    Yes, particularly if that memory is designed and engineered by Apple in house like Apple Silicon in house and manufactured by TSMC on shore somewhere in the United States.
- connicpu 4mo ago
  The Nvidia GB300 DGX Station, which isn't even going to hit 1TB total memory, is expected to launch at almost $100k. Bit of a pipe dream with memory prices where they're at.
  - FuriouslyAdrift 4mo ago
    There are multiple server systems available right around the $100k range that have 512B of GPU RAM right now (4x AMD Instinct MI300A) GIGABYTE G383-R80-AAP1 for example
- bigyabai 4mo ago
  > Apple would basically be able to extinguish the entirety of the market cap for Nvidia I don't think you understand why people buy Nvidia hardware if you're beating the "just add more dual channel DDR, bro" drum. Apple wouldn't even be able to extinguish AMD with a product like that, it's all slow memory being fed into a raster-first GPU architecture.
pstuart 4mo ago
The work on LLM in a Flash will probably help, and Apple's NVMe architecture is well suited to maximize throughput could allow their devices to work better on larger models than other vendors.
- ABS 4mo ago
  [flagged]

satvikpendem 4mo ago

You can use OpenCode or Pi with SSD streaming so it technically will have all the features, just unbearably slow.

jubilanti 4mo ago

> all of the existing features of Claude Code but somehow running locally on my laptop's neural engine You can use environment variables to have claude code query literally any endpoint you choose as long as it has a compatible API.

5701652400 4mo ago

I would not mind if cloud was actually private users iCloud. users pay for it, and it runs in Apple servers next to where users store their iPhotos already. that would be really elegant solution. ..but instead we get Claude, hosted who-knows-where. maybe in X-AI datacenters? maybe in Amazon somewhere? who knows..

willy_k 4mo ago
https://security.apple.com/blog/private-cloud-compute/ https://security.apple.com/blog/private-cloud-compute/