3 ms·

i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundationmodels https://developer

by franze 4mo ago

i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundationmodels https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel https://github.com/Arthur-Ficial/apfel so i might be biased

crancher 4mo ago
Apfel is very useful, thanks for the effort.
- cat5e 4mo ago
  I second this, I’m more excited about dumb local models than something I could never run locally.
trollbridge 4mo ago
Thanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.
robgough 4mo ago
Have you seen that they've added an `fm` tool? It was mentioned in the Platforms State of the Union. Here's what you get when you run it... https://gist.github.com/robgough/7893602895e75801174750761988ffca https://gist.github.com/robgough/7893602895e7580117475076198...
- ElFitz 4mo ago
  Oh, neat. Totally missed it, thanks!
- harrouet 4mo ago
  Wait, is PPC open bar ?
  - robgough 4mo ago
    They've said there are limits, and increased limits for those on iCloud+ ... so it seems that Apple is in the selling LLM access game now. I don't think there are any details yet on the nature of those limits, and whether they can be increased as required etc.
- dofm 4mo ago
  Is ‘fm serve’ OpenAPI compatible?
  - franze 4mo ago
    fm serve - "Start a Chat Completions API server" chat completion is openai's api surface name. but only when it is actually available we will see if it's a clean drop-in vs. just "chat-completions-ish". one of my learnings from apfel is that is is very easy to get a kinda openAI api compatible server, and a lot of work to get it really totally compatible. sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core....
    - dofm 4mo ago
      > chat completion is openai's api surface name Ahh! I did not know that > sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core…. It's a similar situation with "Arca-Swiss compatible" tripod plates in photography. There is really no such thing — Arca-Swiss didn't make a standard, so they didn't have to stick to it themselves, and while most things using this "standard" fix to most things, some things just won't fit, or won't stay put. Everyone implements it, and if they don't, people complain "why didn't you just put an Arca standard foot on it?" and then you have to sit them down and tell them.
- franze 4mo ago
  did miss it until now, cool to see it on device and first party. as soon as it lands I will see the impact on apfel. but i definitely feel flattered, either my little project inspired them or that I reached the same conclusion at a similar time as a team at apple that "hey, this is totally missing"
- ABS 4mo ago
  [dead]
mips_avatar 4mo ago
Seems like they still won’t let you run models on GPU while the phone is closed or the user switches apps
- tyre 4mo ago
  This is good. Apps would not be respectful and end up draining users’ batteries to zero in no time.
dofm 4mo ago
Are you surprised they apparently didn’t adopt your idea and add an OpenAPI compatible endpoint in Core AI, even if just as a testing tool? I am. I also really want to hear more about their containerisation/seatbelt strategy now that they are offering MCP support. Not seen any news about Darwin inside their containers system. (Apfel is a cool project; it’s been the only thing tempting me to upgrade to Tahoe)
mark_l_watson 4mo ago
Thanks apfel looks useful! I have been experimenting with Apple's foundation models for almost a year and they are useful for embedded applications. I have been taking a deeper dive into local agentic coding tools (starting with 'little-coder --model ollama/gemma4:12b-it-qat') and I put together a tiny free book with some setup advice that might save people a few minutes of setup time: https://leanpub.com/read/local-coding-agents https://leanpub.com/read/local-coding-agents I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.
jjice 4mo ago
Agreed. The idea of a system wide (and platform wide) on device model being a core part of OS APIs is very appealing. I do like my software more piecemeal, generally, but when it comes to Apple, I really love a lot of the out-of-the-box offerings they have. Just giving software access to something they know exists on these platforms and can use for various small (and likely increasingly large) gen AI tasks is so appealing.
ABS 4mo ago
[dead]

by franze 4mo ago

crancher 4mo ago

Apfel is very useful, thanks for the effort.

cat5e 4mo ago
I second this, I’m more excited about dumb local models than something I could never run locally.

trollbridge 4mo ago

Thanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.

robgough 4mo ago

Have you seen that they've added an `fm` tool? It was mentioned in the Platforms State of the Union. Here's what you get when you run it... https://gist.github.com/robgough/7893602895e75801174750761988ffca https://gist.github.com/robgough/7893602895e7580117475076198...

ElFitz 4mo ago
Oh, neat. Totally missed it, thanks!
harrouet 4mo ago
Wait, is PPC open bar ?
- robgough 4mo ago
  They've said there are limits, and increased limits for those on iCloud+ ... so it seems that Apple is in the selling LLM access game now. I don't think there are any details yet on the nature of those limits, and whether they can be increased as required etc.
dofm 4mo ago
Is ‘fm serve’ OpenAPI compatible?
- franze 4mo ago
  fm serve - "Start a Chat Completions API server" chat completion is openai's api surface name. but only when it is actually available we will see if it's a clean drop-in vs. just "chat-completions-ish". one of my learnings from apfel is that is is very easy to get a kinda openAI api compatible server, and a lot of work to get it really totally compatible. sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core....
  - dofm 4mo ago
    > chat completion is openai's api surface name Ahh! I did not know that > sometimes i wonder if even the openai implementation of openai's api is openai api compatible to the core…. It's a similar situation with "Arca-Swiss compatible" tripod plates in photography. There is really no such thing — Arca-Swiss didn't make a standard, so they didn't have to stick to it themselves, and while most things using this "standard" fix to most things, some things just won't fit, or won't stay put. Everyone implements it, and if they don't, people complain "why didn't you just put an Arca standard foot on it?" and then you have to sit them down and tell them.
franze 4mo ago
did miss it until now, cool to see it on device and first party. as soon as it lands I will see the impact on apfel. but i definitely feel flattered, either my little project inspired them or that I reached the same conclusion at a similar time as a team at apple that "hey, this is totally missing"
ABS 4mo ago
[dead]

mips_avatar 4mo ago

Seems like they still won’t let you run models on GPU while the phone is closed or the user switches apps

tyre 4mo ago
This is good. Apps would not be respectful and end up draining users’ batteries to zero in no time.

dofm 4mo ago

Are you surprised they apparently didn’t adopt your idea and add an OpenAPI compatible endpoint in Core AI, even if just as a testing tool? I am. I also really want to hear more about their containerisation/seatbelt strategy now that they are offering MCP support. Not seen any news about Darwin inside their containers system. (Apfel is a cool project; it’s been the only thing tempting me to upgrade to Tahoe)

mark_l_watson 4mo ago

Thanks apfel looks useful! I have been experimenting with Apple's foundation models for almost a year and they are useful for embedded applications. I have been taking a deeper dive into local agentic coding tools (starting with 'little-coder --model ollama/gemma4:12b-it-qat') and I put together a tiny free book with some setup advice that might save people a few minutes of setup time: https://leanpub.com/read/local-coding-agents https://leanpub.com/read/local-coding-agents I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.

jjice 4mo ago

Agreed. The idea of a system wide (and platform wide) on device model being a core part of OS APIs is very appealing. I do like my software more piecemeal, generally, but when it comes to Apple, I really love a lot of the out-of-the-box offerings they have. Just giving software access to something they know exists on these platforms and can use for various small (and likely increasingly large) gen AI tasks is so appealing.

ABS 4mo ago

[dead]