9 ms·

Gemma 4 on iPhone

868 points by janandonly 7mo ago

darshil2023 7mo ago
[dead]
hadrien01 7mo ago
Is it me or does the App Store website look... fake? The text in the header ("Productiviteit", "Alleen voor iPhone") looks pixelated, like it was edited on Paint, the header background is flickering, the app icon and screenshots are very low quality, the title of the website is incomplete ("App Store voor iPho...")
- piperswe 7mo ago
  What browser are you using? I don't see any of this behavior on Firefox...
  - hadrien01 7mo ago
    Firefox on Windows, but it looks about the same in Edge Screenshot of the header: https://i.imgur.com/4abfGYF.png https://i.imgur.com/4abfGYF.png
    - morpheuskafka 7mo ago
      It looks like there is some sort of glow effect on the text that isn't rendering right on your browser? It arguably doesn't have the best contrast, but seems to be as intended in Safari 26.3. Looks similar on Chrome macOS too: https://imgur.com/yq5PrKm https://imgur.com/yq5PrKm.
    - t-sauer 7mo ago
      Renders equally weird for me on Firefox on Windows 11. Firefox on MacOS looks good though. Edit: Seems like mix-blend-mode: plus-lighter is bugged in Firefox on Windows https://jsfiddle.net/bjg24hk9/ https://jsfiddle.net/bjg24hk9/
  - OJFord 7mo ago
    Firefox on Android: 'Google AI' (in app name) is clipped off the top; the Apple 'share' button is clipped on the bottom.
- throwatdem12311 7mo ago
  Issues caused by a low effort localization? On my iPhone it opens on the App Store app, so it looks fine to me.
- j0hax 7mo ago
  Everything renders crystal clear with Firefox on GrapheneOS.
- giarc 7mo ago
  It's the dutch version, see /nl/ in the url. If you just go to https://apps.apple.com/ https://apps.apple.com/ it does look better, but I agree, still a bit "off".
- ezfe 7mo ago
  Nothing weird on my side
- lateforwork 7mo ago
  Here's the US version of the same page: https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337 https://apps.apple.com/us/app/google-ai-edge-gallery/id67496... The design quality is still poor. But that's the new Apple. Design is no longer one of their core strengths.
pmarreck 7mo ago
Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic , I was too tempted to try it with this model a couple days ago (made a repo to make it easier, actually) https://github.com/pmarreck/gemma4-heretical https://github.com/pmarreck/gemma4-heretical and... Wow. It worked. And... Not having a built-in nanny is fun! It's also possible to make an MLX version of it, which runs a little faster on Macs, but won't work through Ollama unfortunately. (LM Studio maybe.) Runs great on my M4 Macbook Pro w/128GB and likely also runs fine under 64GB... smaller memories might require lower quantizations. I specifically like dealigned local models because if I have to get my thoughts policed when playing in someone else's playground, like hell am I going to be judged while messing around in my own local open-source one too. And there's a whole set of ethically-justifiable but rule-flagging conversations (loosely categorizable as things like "sensitive", "ethically-borderline-but-productive" or "violating sacred cows") that are now possible with this, and at a level never before possible until now. Note: I tried to hook this one up to OpenClaw and ran into issues To answer the obvious question- Yes, this sort of thing enables bad actors more (as do many other tools). Fortunately, there are far more good actors out there, and bad actors don't listen to rules that good actors subject themselves to, anyway.
- jackp96 7mo ago
  [flagged]
  - potsandpans 7mo ago
    I'm tired of this concern trolling.
    - deleted 7mo ago
      [deleted]
    - jackp96 7mo ago
PullJosh 7mo ago
This is awesome! 1) I am able to run the model on my iPhone and get good results. Not as good as Gemini in the cloud, but good. 2) I love the “mobile actions” tool calls that allow the LLM to turn on the flashlight, open maps, etc. It would be fun if they added Siri Shortcuts support. I want the personal automation that Apple promised but never delivered. 3) I am so excited for local models to be normalized. I build little apps for teachers and there are stringent privacy laws involved that mean I strongly prefer writing code that runs fully client-side when possible. When I develop apps and websites, I want easy API access to on-device models for free. I know it sort of exists on iOS and Chrome right now, but as far as I’m aware it’s not particularly good yet.
- buzzerbetrayed 7mo ago
  For me the hallucination and gaslighting is like taking a step back in time a couple of years. It even fails the “r’s in strawberry” question. How nostalgic. It’s very impressive that this can run locally. And I hope we will continue to be able to run couple-year-old-equivalent models locally going forward.
  - 1f60c 7mo ago
    Strangely, reasoning is not on by default. If you enable it, it answers as you'd expect.
  - dimmke 7mo ago
    I haven't seen anybody else post it in this thread, but this is running on 8GB of RAM. It's not the full Gemma 4 32B model. It's a completely different thing from the full Gemma 4 experience if you were running the flagship model, almost to the point of being misleading. It's their E2B and E4B variants (so 2B and 4B but also quantized) https://ai.google.dev/gemma/docs/core/model_card_4#dense_models https://ai.google.dev/gemma/docs/core/model_card_4#dense_mod...
    - zozbot234 7mo ago
      The relevant constraint when running on a phone is power, not really RAM footprint. Running the tiny E2B/E4B models makes sense, this is essentially what they're designed for.
jeroenhd 7mo ago
English version of the page: https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337 https://apps.apple.com/us/app/google-ai-edge-gallery/id67496... Also on Android: https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery https://play.google.com/store/apps/details?id=com.google.ai.... It's a demo app for Google's Edge project: https://ai.google.dev/edge https://ai.google.dev/edge
- deleted 7mo ago
  [deleted]
- om252345 7mo ago
  Gemma4 works really slow on my android e2b model on Samsung galaxy s21 ultra. Atleast 20-30 sec to warm up and then reply.
  - satvikpendem 7mo ago
    Needs a modern phone, local LLMs don't work well on older phones.
  - cobicobi 7mo ago
    need s24 ultra and above i think
  - thepbone 7mo ago
    The bigger E4B model is pretty fast on my Galaxy S21 Ultra even with thinking enabled. Maybe GPU acceleration was not enabled?
    - jeroenhd 7mo ago
      I think there's quite the performance difference between the S21 Ultra (Snapdragon 888) and the S21 Ultra (Exynos 2100). Qualcomm has optimized libraries for running LLMs on their chips that I don't believe Samsung has bothered with.
  - jeroenhd 7mo ago
deleted 7mo ago
[deleted]
carbocation 7mo ago
It would be very helpful if the chat logs could (optionally) be retained.
TGower 7mo ago
These new models are very impressive. There should be a massive speedup coming as well, AI Edge Gallery is running on GPU, but NPUs in recent high end processors should be much faster. A16 chip for example (Macbook Neo and iphone 16 series) has 35 TOPS of Neural Engine vs 7 TFLOPS gpu. Similar story for Qualcomm.
- api 7mo ago
  That’s nuts actually for such a low power chip. Can’t wait to see the M series version of that. I’m sure very fast TPUs in desktops and phones are coming.
  - zozbot234 7mo ago
    The Apple Silicon in the MacBook Neo is effectively a slimmed down version of M4, which is already out and has a very similar NPU (similar TFLOPS rating). It's worth noting however that the TFLOPS rating for Apple Neural Engine is somewhat artificial, since e.g. the "38 TFLOPS" in the M4 ANE are really 19 TFLOPS for FP16-only operation.
janandonly 7mo ago
OP Here. It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive then it is today. The latter option will only bemusedly for tasks that humans are more expensive or much slower in. This Gemma 4 model gives me hope for a future Siri or other with iPhone and macOS integration, “Her” (as in the movie) style.
- kennywinker 7mo ago
  Did you really watch “Her” and think this is a future that should happen?? Seriously????
  - deleted 7mo ago
    [deleted]
  - jfreds 7mo ago
    I don’t think OP’s point has anything to do with AI companions. The big benefit of moving compute to edge devices is to distribute the inference load on the grid. Powering and cooling phones is a lot easier than powering and cooling a datacenter
    - kennywinker 7mo ago
      Local ai is probably a good direction, i agree. But there was a part of their point that had to do with ai companions: the bit where they say we are closer to “her”-like ai companions. That was the bit i was responding to.
  - sambapa 7mo ago
    Torment Nexus sounds fun
    - kennywinker 7mo ago
      Watch out! We got an info hazard here! Danger danger
dwa3592 7mo ago
I think with this google starts a new race- best local model that runs on phones.
- dwa3592 7mo ago
  I wonder why the cut off date for 3n-E4B-it is Oct, 2023. That's really far in the past.
  - satvikpendem 7mo ago
    Because that's Gemma 3, not 4.
burnto 7mo ago
My iPhone 13 can’t run most of these models. A decent local LLM is one of the few reasons I can imagine actually upgrading earlier than typically necessary.
- Gigachad 7mo ago
  I’ve got a 17 pro and tbh I haven’t found any use for local models yet. They are a neat curiosity but the online ones are absolutely massively far ahead. Considering they are being given away for free currently, it’s hard to justify not making use of them over dumber local models.
- mchusma 7mo ago
  I’m expecting the new iPhone release this fall to be coupled with some great version of Siri/model. This could be the first reason I’ve seen to upgrade in a while (although even that I’m not sure of, as I am king of in the “always use the best model it’s worth it” camp.) Apple has a great shot at making a highly optimized 4.5 version of this model highly tuned to the next gen iPhone, which could work great.
deckar01 7mo ago
It doesn’t render Markdown or LaTeX. The scrolling is unusable during generation. E4B failed to correctly account for convection and conduction when reasoning about the effects of thermal radiation (31b was very good). After 3 questions in a session (with thinking) E4B went off the rails and started emitting nonsense fragment before the stated token limit was hit (unless it isn’t actually checking).
- 3abiton 7mo ago
  They have very limited capabilities compared to bigger more complex models, but for general stuff, they are fantastic. We need to set the expectations correctly of what they can do, I know lots of hype around Gemma 4, even though Qwen3.5 outperformed it. It's just a reliable overall small model, with great small model abilities.
__natty__ 7mo ago
That's a great project! I just wondered whether Google would have a problem with you using their trademark
- tech234a 7mo ago
  This is an app published by Google itself
rickdg 7mo ago
How do these compare to Apple's Foundation Models, btw?
- simonw 7mo ago
  So much better. Hard to quantify, but even the small Gemma 4 models have that feels-like-ChatGPT magic that Apple's models are lacking.
- snarkyturtle 7mo ago
  AFM had a 4096 token context window and this can be configured to have a 32k+ token context window, for one.
karimf 7mo ago
This app is cool and it showcases some use cases, but it still undersells what the E2B model can do. I just made a real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B. I posted it on /r/LocalLLaMA a few hours ago and it's gaining some traction [0]. Here's the repo [1] I'm running it on a Macbook instead of an iPhone, but based on the benchmark here [2], you should be able to run the same thing on an iPhone 17 Pro. [0] https://www.reddit.com/r/LocalLLaMA/comments/1sda3r6/realtime_ai_audiovideo_in_voice_out_on_an_m3_pro/ https://www.reddit.com/r/LocalLLaMA/comments/1sda3r6/realtim... [1] https://github.com/fikrikarim/parlor https://github.com/fikrikarim/parlor [2] https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm https://huggingface.co/litert-community/gemma-4-E2B-it-liter...
- nothinkjustai 7mo ago
  Parlor is so cool, especially since you’re offering it for free. And a great use case for local LLMs.
  - karimf 7mo ago
    Thanks! Although, I can't claim any credit for it. I just spent a day gluing what other people have built. Huge props to the Gemma team for building an amazing model and also an inference engine that's focused for edge devices [0] [0] https://github.com/google-ai-edge/LiteRT-LM https://github.com/google-ai-edge/LiteRT-LM
- storus 7mo ago
  That's cool! You can add SoulX-FlashHead for real-time AI head animation as well if you want to simulate a teacher.
  - karimf 7mo ago
    Thanks for sharing! I'm still torn about it. Sure it'll feel more natural if you have the AI head animation, but I don't want people to get attached to it. I don't want to make the loneliness epidemic even worse.
- 7mo ago
beeflet 7mo ago
Isn't this already possible in a much more open-ended way with PocketPal? https://github.com/a-ghorbani/pocketpal-ai https://github.com/a-ghorbani/pocketpal-ai https://apps.apple.com/us/app/pocketpal-ai/id6502579498 https://apps.apple.com/us/app/pocketpal-ai/id6502579498 https://play.google.com/store/apps/details?id=com.pocketpalai https://play.google.com/store/apps/details?id=com.pocketpala...
- lzzqrd 7mo ago
  Could you clarify what you mean by 'open-ended' in this context, since both initiatives are essentially open-source?
dzhiurgis 7mo ago
I recently got to a first practical use of it. I was on a plane, filling landing card (what a silly thing these are). I looked up my hotel address using qwen model on my iPhone 16 Pro. It was accurate. I was quite impressed. After some back and forth the chat app started to crash tho, so YMMV.
meidad_g 7mo ago
[flagged]
allpratik 7mo ago
Nice! Tried on iPhone 16 pro with 30 TPS from Gemma-4-E2B-it model. Although the phone got considerably hot while inferencing. It’s quite an impressive performance and cannot wait to try it myself in one of my personal apps.
- golem14 7mo ago
  It's at least somewhat limited in non-English content. It knows how to make lentil soup, so I was happy that I never need to look up recipe sites with awful UX and ads, but then it couldn't find a recipe for "Kalter Hund"/"Kalte Schnauze". So sad ;) Still, absolutely fabulous. What a time to be alive!
- mudkipdev 7mo ago
  It's strange that my iPhone 14 is at regular temperature when using the E2B model. But also it's a lot slower (not sure how to measure the exact tokens per second, ~12 if I had to guess)
garff 7mo ago
How new of an iPhone model is needed?
XCSme 7mo ago
Gemma 4 is great: https://aibenchy.com/compare/google-gemma-4-31b-it-medium/google-gemma-4-26b-a4b-it-medium/google-gemini-3-pro-preview-medium/ https://aibenchy.com/compare/google-gemma-4-31b-it-medium/go... I assume it is the 26B A4B one, if it runs locally?
- adrian17 7mo ago
  No, only E2B and E4B.
dhbradshaw 7mo ago
My son just started using 2B on his Android. I mentioned that it was an impressively compact model and next thing I knew he had figured out how to use it on his inexpensive 2024 Motorolla and was using it to practice reading and writing in foreign languages.
thot_experiment 7mo ago
Gemma 4 E4B is an incredible model for doing all the home assistant stuff I normally just used Qwen3.5 35BA4B + Whisper while leaving me with wayy more empty vram for other bullshit. It works as a drop in replacement for all of my "turn the lights off" or "when's the next train" type queries and does a good job of tool use. This is the really the first time vramlets get a model that's reliably day to day useful locally. I'm curious/worried about the audio capability, I'm still using Whisper as the audio support hasn't landed in llama.cpp, and I'm not excited enough to temporarily rewire my stuff to use vLLM or whatever their reference impl is. The vision capabilities of Gemma are notably (thus far, could be impl specific issues?) much much worse than Qwen (even the big moe and dense gemma are much worse), hopefully the audio is at least on par with medium whisper.
lol8675309 7mo ago
It’s gotta be free!?!? Right!?!? Oh oh wait
neurostimulant 7mo ago
I'm able to sweet talk the gemma-4-e2b-it model in an iphone 15 to solve a hcaptcha screenshot. This small model is surprisingly very capable!
mc7alazoun 7mo ago
Would it work locally on a Mac Pro M4 24gb? If so I'd really appreciate a step-by-step guide.
- weberer 7mo ago
  These E2B and E4B models are very small so that they can fit into phones with around 8gb of RAM. You can get away with a much larger model. Just run: brew install ollama ollama run gemma4:26b-a4b-it-q4_K_M
  - mc7alazoun 7mo ago
    Legend! Thanks heaps.
rotexo 7mo ago
E4B is pretty good for extracting tables of items from receipt scans and inferring categories, wish this could be called from within a shortcut to just select a photo and add the extracted table to the clipboard
Waterluvian 7mo ago
I see a phenomenal opportunity for old phone re-use by arraying them in some dock and making them be my "home AI."
tithos 7mo ago
Most of the models are not available. I’m guessing they will become available soon enough… At least I hope.
yalogin 7mo ago
Are these models open source? If so this is Google’s attempt to collect user data from their models.
- int_19h 7mo ago
  How is Google going to collect user data from a locally running model?
  - yalogin 7mo ago
    If you do it yourself they don’t, that is why they are packaging into an app
micmcfly 7mo ago
[dead]
satvikpendem 7mo ago
This is also on Android and has an option to use AICore with the NPU which can run much faster than even the GPU models.
- nout 7mo ago
  How do you get it running on Android?
  - satvikpendem 7mo ago
    It's the same app, Google AI edge gallery.
davecahill 7mo ago
I really like Enclave for on-device models - looks like they're about to add Gemma 4 too: https://enclaveai.app/blog/2026/04/02/gemma-4-release-on-device-local-ai/ https://enclaveai.app/blog/2026/04/02/gemma-4-release-on-dev...
- robbru 7mo ago
  I've been using Enclave ever since, they have been the best App Store option for a long time.
rudedogg 7mo ago
This is fun, FYI you don’t have to sign in/up with a Google account. I hesitated downloading it for that reason.
two_handfuls 7mo ago
The description says it's private, but the legalese it makes you agree to makes no promise. Rather, the opposite: > We collect information about your activity in our services Source: https://policies.google.com/privacy#infocollect https://policies.google.com/privacy#infocollect
- bigyabai 7mo ago
  The app is open source[0], although given Apple's stance on sideloading it's hard to confirm if you're using the open version. [0] https://github.com/google-ai-edge/gallery https://github.com/google-ai-edge/gallery
  - selfsigned 7mo ago
    Two (very quick) minutes on their GitHub repo and it's pretty obvious that they're using firebase-analytics and at the very least seem to be sending URLs[1] and infos such as the model you download or the capacities[2] you use. [1] https://github.com/google-ai-edge/gallery/blob/main/Android/src/app/src/main/java/com/google/ai/edge/gallery/ui/common/ClickableLink.kt#L62 https://github.com/google-ai-edge/gallery/blob/main/Android/... [2] https://github.com/google-ai-edge/gallery/blob/main/Android/src/app/src/main/java/com/google/ai/edge/gallery/customtasks/mobileactions/MobileActionsScreen.kt#L413 https://github.com/google-ai-edge/gallery/blob/main/Android/...
    - kaliqt 7mo ago
      That’s not the same app as posted which is for iOS.
      - selfsigned 7mo ago
        You're right, they reference IOS in the README.md and link to the GitHub repo from the Apple Store page yet don't include the IOS source, sneaky.
  - kaliqt 7mo ago
    That is the Android repo, where is the iOS repo?
Sharmaji000 7mo ago
Still didnt release training recipe, data, methodology etc unlike deepseek. Mostly released to get developer ecosystem across their android built in ai. Still good and interesting, but not exactly philanthropic to the open source progress.
nickvec 7mo ago
Extremely impressed by how fast responses are on iPhone 17 Pro Max. Can’t wait for this to be used for Siri’s brain one of these days (hopefully!)
al_borland 7mo ago
I find it odd they are using the term “edge” to brand this, if it’s target is the general public. I’ve been to a few tech conferences and saw the term used there for the first time. It took me a little bit to see the pattern and understand what it meant. I have never heard the term used outside of those circles. It seems like “local” would be the term average users would be familiar with. Normal people don’t call their stuff “edge devices”.
- deleted 7mo ago
  [deleted]
- bigyabai 7mo ago
  > if it’s target is the general public. It's not - Apple is working with Google right now to make Siri into the public-facing version of this. This is kinda just the tech preview before all the branding has been painted on.
- ycombinete 7mo ago
  Funnily enough I work in the security industry and the term is ubiquitous there, so I didn’t even notice it.
danielrmay 7mo ago
I spent some time getting Gemma4-e4b working via llamacpp on iPhone and I'm really impressed so far! I posted a short video of an example application on LinkedIn here https://www.linkedin.com/feed/update/urn:li:activity:7446746072569491456/ https://www.linkedin.com/feed/update/urn:li:activity:7446746... (or x: https://x.com/danielrmay/status/2040971117419192553 https://x.com/danielrmay/status/2040971117419192553)
MysticOracle 7mo ago
Crashes for me on a couple of different iDevices (2 generations behind) after only a few 2-3 chats. Probably not enough RAM. Saw this one on X the other day updated with Gemma 4 and they have the built-in Apple Foundation model, Qwen3.5, and other models: Locally AI - https://locallyai.app/ https://locallyai.app/
LeonTing1010 7mo ago
[flagged]
rock_artist 7mo ago
I really believe in the future of local models. From app developer and user, My main concern for now is bloating devices. Until we’ll have something like Apples foundation model where multiple apps could share the same model it means we have something horrible as Electron in the sense, every app is a fully blown model (browser in the electron story) instead of reusing the model. With desktops we have DLL hell for years. But with sandboxed apps on mobile devices it becomes a bigger issue that I guess will/should be addressed by the OS. For my app I’ve been trying to add some logic based on large model but for bloating a simple Swift app with 2-3GB of model or even few hundred MBs feels wrong doing and conflicting with code reusability concepts.
- janandonly 7mo ago
  This app unlocks using the Apple Foundation model itself: https://apps.apple.com/nl/app/locally-ai-local-ai-chat/id6741426692 https://apps.apple.com/nl/app/locally-ai-local-ai-chat/id674...
- hypersolo 7mo ago
  [flagged]
modeless 7mo ago
It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model. Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.
- ulfw 7mo ago
  It runs very fast on my Qualcomm Elite Gen 5 SoC Oppo Find N6
  - allpratik 7mo ago
    How many tokens per second? Also, does it get warm/hot?
    - modeless 7mo ago
      If this Gemma tokenizer I found online is accurate then my Pixel 10 Pro XL is getting ~22 tok/s on Gemma 4 E2B using the NPU, vs. 40 tok/s is what people are saying the MLX version gets on iPhone. Actually I found official performance numbers from Google saying iPhone gets 56 tok/s and Qualcomm gets 52. They don't even bother listing Tensor in their table. Maybe because it would be too embarrassing. Ouch! https://ai.google.dev/edge/litert-lm/overview https://ai.google.dev/edge/litert-lm/overview
jdthedisciple 7mo ago
it's Google, so is it really private? remember, megacorps are dying for infinite amounts of analytics data
- classified 7mo ago
  When I saw it wants me to "agree" to Google's "privacy policy", I deleted the app on the spot.
- prism56 7mo ago
  Are there any alternatives for on device android llm that aren't google and/or more private?
rcarmo 7mo ago
This is fun. I just wish I could add more skills, the UX is too dumbed down but knowing there is a run_js tool there is a lot that can be done here.
_nagu_ 7mo ago
If this works smoothly on iPhone, it could change how we think about mobile apps. Less backend dependency, more on-device intelligence.
- jcutrell 7mo ago
  This is what Apple promised a long time ago, and just couldn't quite connect on delivery.
totetsu 7mo ago
I have been looking at ARGmax https://www.argmaxinc.com/#SDK https://www.argmaxinc.com/#SDK for running on apple devices, but not sure yet at whats involved in porting a model to work with their sdk
gdzie-jest-sol 7mo ago
I need normal server too in local network I can run chat in other device and 'counting' on iphone. Second idea is input audio in other language, like Czech, Polish, French
orf 7mo ago
I’d recommend locally.ai[1] - it’s really good and has a wide range of models. Also has shortcuts support. 1. https://apps.apple.com/gb/app/locally-ai-local-ai-chat/id6741426692 https://apps.apple.com/gb/app/locally-ai-local-ai-chat/id674...
- janandonly 7mo ago
  Thanks for the link. Gemma 4 also works in this app.
imadselka 7mo ago
good model!
nightrate_ai 7mo ago
[dead]
MagicMoonlight 7mo ago
It seems really capable. A few more iterations of this and you won’t even need a subscription. All it needs is web search so that it can get up to date information.
derwiki 7mo ago
I asked it about the “Altamont Free Concert” (exact name of Wikipedia article), and it’s been a while since I’ve seen an hallucination this rich. Doesn’t give me confidence to use it.
Sukhesh-QA 7mo ago
[dead]
areys 7mo ago
[flagged]
haizhung 7mo ago
I encourage everybody to try this, if they have an iPhone. If you’re like me and don’t have the time to tinker with the latest and greatest all the time; this app lowers the barrier to entry significantly and provides a glimpse into what’s possible locally, on device. Honestly, I was extremely impressed by the speed and quality of the answers considering this thing runs on a phone. It honestly makes me want to sit down and spin up my own homegrown AI setup to go fully independent. Crazy.
ValveFan6969 7mo ago
[dead]
amai 7mo ago
The cooperation of Apple and Google is going to crush the competition: https://blog.google/company-news/inside-google/company-announcements/joint-statement-google-apple/ https://blog.google/company-news/inside-google/company-annou... The combination of Apples hardware and Googles software is unbeatable.
- bigyabai 7mo ago
  With Google's graveyard and Apple's walled garden, nothing can stop the enshittification train from trundling down the tracks.
- lemonish97 7mo ago
  Isn't their competition (at least in the mobile space) google/android themselves?
sshrajesh 7mo ago
> Note: I tried to hook this one up to OpenClaw and ran into issues Anyone worked on hooking up OpenClaw to gemma4 running locally?
areys 7mo ago
[flagged]
lemonish97 7mo ago
I hope they add a web search tool to the agent skills too. Most of my llm usage on my phone are just quick lookups and search summarizations. Would love to do these with a local model rather than Google AI mode of any other cloud based inference tools.
inzlab 7mo ago
Impressive
pseudosavant 7mo ago
It'd be fun to explore creating a Gemma 4 LLM API server app so you could use your iPhone's processing for agentic coding on a laptop. I don't know how useful it would be, but it'd be fun.
linncharm 7mo ago
[dead]
aimemobe 6mo ago
[flagged]
deleted 6mo ago
[deleted]
znnajdla 6mo ago
One really good use case for me is good, fast, offline translation. Both Apple Translate and Google Translate are worse in quality than a decent LLM and don’t work well offline. Gemma 4 is surprisingly good and often faster than waiting for an API call.

868 points by janandonly 7mo ago

darshil2023 7mo ago

[dead]

Is it me or does the App Store website look... fake? The text in the header ("Productiviteit", "Alleen voor iPhone") looks pixelated, like it was edited on Paint, the header background is flickering, the app icon and screenshots are very low quality, the title of the website is incomplete ("App Store voor iPho...")

piperswe 7mo ago
What browser are you using? I don't see any of this behavior on Firefox...
- hadrien01 7mo ago
  Firefox on Windows, but it looks about the same in Edge Screenshot of the header: https://i.imgur.com/4abfGYF.png https://i.imgur.com/4abfGYF.png
  - morpheuskafka 7mo ago
    It looks like there is some sort of glow effect on the text that isn't rendering right on your browser? It arguably doesn't have the best contrast, but seems to be as intended in Safari 26.3. Looks similar on Chrome macOS too: https://imgur.com/yq5PrKm https://imgur.com/yq5PrKm.
  - t-sauer 7mo ago
    Renders equally weird for me on Firefox on Windows 11. Firefox on MacOS looks good though. Edit: Seems like mix-blend-mode: plus-lighter is bugged in Firefox on Windows https://jsfiddle.net/bjg24hk9/ https://jsfiddle.net/bjg24hk9/
- OJFord 7mo ago
  Firefox on Android: 'Google AI' (in app name) is clipped off the top; the Apple 'share' button is clipped on the bottom.
throwatdem12311 7mo ago
Issues caused by a low effort localization? On my iPhone it opens on the App Store app, so it looks fine to me.
j0hax 7mo ago
Everything renders crystal clear with Firefox on GrapheneOS.
giarc 7mo ago
It's the dutch version, see /nl/ in the url. If you just go to https://apps.apple.com/ https://apps.apple.com/ it does look better, but I agree, still a bit "off".
ezfe 7mo ago
Nothing weird on my side
lateforwork 7mo ago
Here's the US version of the same page: https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337 https://apps.apple.com/us/app/google-ai-edge-gallery/id67496... The design quality is still poor. But that's the new Apple. Design is no longer one of their core strengths.

pmarreck 7mo ago

Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?) I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic https://github.com/p-e-w/heretic , I was too tempted to try it with this model a couple days ago (made a repo to make it easier, actually) https://github.com/pmarreck/gemma4-heretical https://github.com/pmarreck/gemma4-heretical and... Wow. It worked. And... Not having a built-in nanny is fun! It's also possible to make an MLX version of it, which runs a little faster on Macs, but won't work through Ollama unfortunately. (LM Studio maybe.) Runs great on my M4 Macbook Pro w/128GB and likely also runs fine under 64GB... smaller memories might require lower quantizations. I specifically like dealigned local models because if I have to get my thoughts policed when playing in someone else's playground, like hell am I going to be judged while messing around in my own local open-source one too. And there's a whole set of ethically-justifiable but rule-flagging conversations (loosely categorizable as things like "sensitive", "ethically-borderline-but-productive" or "violating sacred cows") that are now possible with this, and at a level never before possible until now. Note: I tried to hook this one up to OpenClaw and ran into issues To answer the obvious question- Yes, this sort of thing enables bad actors more (as do many other tools). Fortunately, there are far more good actors out there, and bad actors don't listen to rules that good actors subject themselves to, anyway.

jackp96 7mo ago
[flagged]
- potsandpans 7mo ago
  I'm tired of this concern trolling.
  - deleted 7mo ago
    [deleted]
  - jackp96 7mo ago

PullJosh 7mo ago

This is awesome! 1) I am able to run the model on my iPhone and get good results. Not as good as Gemini in the cloud, but good. 2) I love the “mobile actions” tool calls that allow the LLM to turn on the flashlight, open maps, etc. It would be fun if they added Siri Shortcuts support. I want the personal automation that Apple promised but never delivered. 3) I am so excited for local models to be normalized. I build little apps for teachers and there are stringent privacy laws involved that mean I strongly prefer writing code that runs fully client-side when possible. When I develop apps and websites, I want easy API access to on-device models for free. I know it sort of exists on iOS and Chrome right now, but as far as I’m aware it’s not particularly good yet.

buzzerbetrayed 7mo ago
For me the hallucination and gaslighting is like taking a step back in time a couple of years. It even fails the “r’s in strawberry” question. How nostalgic. It’s very impressive that this can run locally. And I hope we will continue to be able to run couple-year-old-equivalent models locally going forward.
- 1f60c 7mo ago
  Strangely, reasoning is not on by default. If you enable it, it answers as you'd expect.
- dimmke 7mo ago
  I haven't seen anybody else post it in this thread, but this is running on 8GB of RAM. It's not the full Gemma 4 32B model. It's a completely different thing from the full Gemma 4 experience if you were running the flagship model, almost to the point of being misleading. It's their E2B and E4B variants (so 2B and 4B but also quantized) https://ai.google.dev/gemma/docs/core/model_card_4#dense_models https://ai.google.dev/gemma/docs/core/model_card_4#dense_mod...
  - zozbot234 7mo ago
    The relevant constraint when running on a phone is power, not really RAM footprint. Running the tiny E2B/E4B models makes sense, this is essentially what they're designed for.

jeroenhd 7mo ago

English version of the page: https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337 https://apps.apple.com/us/app/google-ai-edge-gallery/id67496... Also on Android: https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery https://play.google.com/store/apps/details?id=com.google.ai.... It's a demo app for Google's Edge project: https://ai.google.dev/edge https://ai.google.dev/edge

deleted 7mo ago
[deleted]
om252345 7mo ago
Gemma4 works really slow on my android e2b model on Samsung galaxy s21 ultra. Atleast 20-30 sec to warm up and then reply.
- satvikpendem 7mo ago
  Needs a modern phone, local LLMs don't work well on older phones.
- cobicobi 7mo ago
  need s24 ultra and above i think
- thepbone 7mo ago
  The bigger E4B model is pretty fast on my Galaxy S21 Ultra even with thinking enabled. Maybe GPU acceleration was not enabled?
  - jeroenhd 7mo ago
    I think there's quite the performance difference between the S21 Ultra (Snapdragon 888) and the S21 Ultra (Exynos 2100). Qualcomm has optimized libraries for running LLMs on their chips that I don't believe Samsung has bothered with.
- jeroenhd 7mo ago

deleted 7mo ago

[deleted]

carbocation 7mo ago

It would be very helpful if the chat logs could (optionally) be retained.

TGower 7mo ago

These new models are very impressive. There should be a massive speedup coming as well, AI Edge Gallery is running on GPU, but NPUs in recent high end processors should be much faster. A16 chip for example (Macbook Neo and iphone 16 series) has 35 TOPS of Neural Engine vs 7 TFLOPS gpu. Similar story for Qualcomm.

api 7mo ago
That’s nuts actually for such a low power chip. Can’t wait to see the M series version of that. I’m sure very fast TPUs in desktops and phones are coming.
- zozbot234 7mo ago
  The Apple Silicon in the MacBook Neo is effectively a slimmed down version of M4, which is already out and has a very similar NPU (similar TFLOPS rating). It's worth noting however that the TFLOPS rating for Apple Neural Engine is somewhat artificial, since e.g. the "38 TFLOPS" in the M4 ANE are really 19 TFLOPS for FP16-only operation.

janandonly 7mo ago

OP Here. It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive then it is today. The latter option will only bemusedly for tasks that humans are more expensive or much slower in. This Gemma 4 model gives me hope for a future Siri or other with iPhone and macOS integration, “Her” (as in the movie) style.

kennywinker 7mo ago
Did you really watch “Her” and think this is a future that should happen?? Seriously????
- deleted 7mo ago
  [deleted]
- jfreds 7mo ago
  I don’t think OP’s point has anything to do with AI companions. The big benefit of moving compute to edge devices is to distribute the inference load on the grid. Powering and cooling phones is a lot easier than powering and cooling a datacenter
  - kennywinker 7mo ago
    Local ai is probably a good direction, i agree. But there was a part of their point that had to do with ai companions: the bit where they say we are closer to “her”-like ai companions. That was the bit i was responding to.
- sambapa 7mo ago
  Torment Nexus sounds fun
  - kennywinker 7mo ago
    Watch out! We got an info hazard here! Danger danger

dwa3592 7mo ago

I think with this google starts a new race- best local model that runs on phones.

dwa3592 7mo ago
I wonder why the cut off date for 3n-E4B-it is Oct, 2023. That's really far in the past.
- satvikpendem 7mo ago
  Because that's Gemma 3, not 4.

burnto 7mo ago

My iPhone 13 can’t run most of these models. A decent local LLM is one of the few reasons I can imagine actually upgrading earlier than typically necessary.

Gigachad 7mo ago
I’ve got a 17 pro and tbh I haven’t found any use for local models yet. They are a neat curiosity but the online ones are absolutely massively far ahead. Considering they are being given away for free currently, it’s hard to justify not making use of them over dumber local models.
mchusma 7mo ago
I’m expecting the new iPhone release this fall to be coupled with some great version of Siri/model. This could be the first reason I’ve seen to upgrade in a while (although even that I’m not sure of, as I am king of in the “always use the best model it’s worth it” camp.) Apple has a great shot at making a highly optimized 4.5 version of this model highly tuned to the next gen iPhone, which could work great.

deckar01 7mo ago

It doesn’t render Markdown or LaTeX. The scrolling is unusable during generation. E4B failed to correctly account for convection and conduction when reasoning about the effects of thermal radiation (31b was very good). After 3 questions in a session (with thinking) E4B went off the rails and started emitting nonsense fragment before the stated token limit was hit (unless it isn’t actually checking).

3abiton 7mo ago
They have very limited capabilities compared to bigger more complex models, but for general stuff, they are fantastic. We need to set the expectations correctly of what they can do, I know lots of hype around Gemma 4, even though Qwen3.5 outperformed it. It's just a reliable overall small model, with great small model abilities.

__natty__ 7mo ago

That's a great project! I just wondered whether Google would have a problem with you using their trademark

tech234a 7mo ago
This is an app published by Google itself

rickdg 7mo ago

How do these compare to Apple's Foundation Models, btw?

simonw 7mo ago
So much better. Hard to quantify, but even the small Gemma 4 models have that feels-like-ChatGPT magic that Apple's models are lacking.
snarkyturtle 7mo ago
AFM had a 4096 token context window and this can be configured to have a 32k+ token context window, for one.

karimf 7mo ago

This app is cool and it showcases some use cases, but it still undersells what the E2B model can do. I just made a real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B. I posted it on /r/LocalLLaMA a few hours ago and it's gaining some traction [0]. Here's the repo [1] I'm running it on a Macbook instead of an iPhone, but based on the benchmark here [2], you should be able to run the same thing on an iPhone 17 Pro. [0] https://www.reddit.com/r/LocalLLaMA/comments/1sda3r6/realtime_ai_audiovideo_in_voice_out_on_an_m3_pro/ https://www.reddit.com/r/LocalLLaMA/comments/1sda3r6/realtim... [1] https://github.com/fikrikarim/parlor https://github.com/fikrikarim/parlor [2] https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm https://huggingface.co/litert-community/gemma-4-E2B-it-liter...

nothinkjustai 7mo ago
Parlor is so cool, especially since you’re offering it for free. And a great use case for local LLMs.
- karimf 7mo ago
  Thanks! Although, I can't claim any credit for it. I just spent a day gluing what other people have built. Huge props to the Gemma team for building an amazing model and also an inference engine that's focused for edge devices [0] [0] https://github.com/google-ai-edge/LiteRT-LM https://github.com/google-ai-edge/LiteRT-LM
storus 7mo ago
That's cool! You can add SoulX-FlashHead for real-time AI head animation as well if you want to simulate a teacher.
- karimf 7mo ago
  Thanks for sharing! I'm still torn about it. Sure it'll feel more natural if you have the AI head animation, but I don't want people to get attached to it. I don't want to make the loneliness epidemic even worse.
7mo ago

beeflet 7mo ago

Isn't this already possible in a much more open-ended way with PocketPal? https://github.com/a-ghorbani/pocketpal-ai https://github.com/a-ghorbani/pocketpal-ai https://apps.apple.com/us/app/pocketpal-ai/id6502579498 https://apps.apple.com/us/app/pocketpal-ai/id6502579498 https://play.google.com/store/apps/details?id=com.pocketpalai https://play.google.com/store/apps/details?id=com.pocketpala...

lzzqrd 7mo ago
Could you clarify what you mean by 'open-ended' in this context, since both initiatives are essentially open-source?

dzhiurgis 7mo ago

I recently got to a first practical use of it. I was on a plane, filling landing card (what a silly thing these are). I looked up my hotel address using qwen model on my iPhone 16 Pro. It was accurate. I was quite impressed. After some back and forth the chat app started to crash tho, so YMMV.

meidad_g 7mo ago

[flagged]

allpratik 7mo ago

Nice! Tried on iPhone 16 pro with 30 TPS from Gemma-4-E2B-it model. Although the phone got considerably hot while inferencing. It’s quite an impressive performance and cannot wait to try it myself in one of my personal apps.

golem14 7mo ago
It's at least somewhat limited in non-English content. It knows how to make lentil soup, so I was happy that I never need to look up recipe sites with awful UX and ads, but then it couldn't find a recipe for "Kalter Hund"/"Kalte Schnauze". So sad ;) Still, absolutely fabulous. What a time to be alive!
mudkipdev 7mo ago
It's strange that my iPhone 14 is at regular temperature when using the E2B model. But also it's a lot slower (not sure how to measure the exact tokens per second, ~12 if I had to guess)

garff 7mo ago

How new of an iPhone model is needed?

XCSme 7mo ago

Gemma 4 is great: https://aibenchy.com/compare/google-gemma-4-31b-it-medium/google-gemma-4-26b-a4b-it-medium/google-gemini-3-pro-preview-medium/ https://aibenchy.com/compare/google-gemma-4-31b-it-medium/go... I assume it is the 26B A4B one, if it runs locally?

adrian17 7mo ago
No, only E2B and E4B.

dhbradshaw 7mo ago

My son just started using 2B on his Android. I mentioned that it was an impressively compact model and next thing I knew he had figured out how to use it on his inexpensive 2024 Motorolla and was using it to practice reading and writing in foreign languages.

thot_experiment 7mo ago

Gemma 4 E4B is an incredible model for doing all the home assistant stuff I normally just used Qwen3.5 35BA4B + Whisper while leaving me with wayy more empty vram for other bullshit. It works as a drop in replacement for all of my "turn the lights off" or "when's the next train" type queries and does a good job of tool use. This is the really the first time vramlets get a model that's reliably day to day useful locally. I'm curious/worried about the audio capability, I'm still using Whisper as the audio support hasn't landed in llama.cpp, and I'm not excited enough to temporarily rewire my stuff to use vLLM or whatever their reference impl is. The vision capabilities of Gemma are notably (thus far, could be impl specific issues?) much much worse than Qwen (even the big moe and dense gemma are much worse), hopefully the audio is at least on par with medium whisper.

lol8675309 7mo ago

It’s gotta be free!?!? Right!?!? Oh oh wait

neurostimulant 7mo ago

I'm able to sweet talk the gemma-4-e2b-it model in an iphone 15 to solve a hcaptcha screenshot. This small model is surprisingly very capable!

mc7alazoun 7mo ago

Would it work locally on a Mac Pro M4 24gb? If so I'd really appreciate a step-by-step guide.

weberer 7mo ago
These E2B and E4B models are very small so that they can fit into phones with around 8gb of RAM. You can get away with a much larger model. Just run: brew install ollama ollama run gemma4:26b-a4b-it-q4_K_M
- mc7alazoun 7mo ago
  Legend! Thanks heaps.

rotexo 7mo ago

E4B is pretty good for extracting tables of items from receipt scans and inferring categories, wish this could be called from within a shortcut to just select a photo and add the extracted table to the clipboard

Waterluvian 7mo ago

I see a phenomenal opportunity for old phone re-use by arraying them in some dock and making them be my "home AI."

tithos 7mo ago

Most of the models are not available. I’m guessing they will become available soon enough… At least I hope.

yalogin 7mo ago

Are these models open source? If so this is Google’s attempt to collect user data from their models.

int_19h 7mo ago
How is Google going to collect user data from a locally running model?
- yalogin 7mo ago
  If you do it yourself they don’t, that is why they are packaging into an app

micmcfly 7mo ago

[dead]

satvikpendem 7mo ago

This is also on Android and has an option to use AICore with the NPU which can run much faster than even the GPU models.

nout 7mo ago
How do you get it running on Android?
- satvikpendem 7mo ago
  It's the same app, Google AI edge gallery.

davecahill 7mo ago

I really like Enclave for on-device models - looks like they're about to add Gemma 4 too: https://enclaveai.app/blog/2026/04/02/gemma-4-release-on-device-local-ai/ https://enclaveai.app/blog/2026/04/02/gemma-4-release-on-dev...

robbru 7mo ago
I've been using Enclave ever since, they have been the best App Store option for a long time.

rudedogg 7mo ago

This is fun, FYI you don’t have to sign in/up with a Google account. I hesitated downloading it for that reason.

two_handfuls 7mo ago

The description says it's private, but the legalese it makes you agree to makes no promise. Rather, the opposite: > We collect information about your activity in our services Source: https://policies.google.com/privacy#infocollect https://policies.google.com/privacy#infocollect

bigyabai 7mo ago
The app is open source[0], although given Apple's stance on sideloading it's hard to confirm if you're using the open version. [0] https://github.com/google-ai-edge/gallery https://github.com/google-ai-edge/gallery
- selfsigned 7mo ago
  Two (very quick) minutes on their GitHub repo and it's pretty obvious that they're using firebase-analytics and at the very least seem to be sending URLs[1] and infos such as the model you download or the capacities[2] you use. [1] https://github.com/google-ai-edge/gallery/blob/main/Android/src/app/src/main/java/com/google/ai/edge/gallery/ui/common/ClickableLink.kt#L62 https://github.com/google-ai-edge/gallery/blob/main/Android/... [2] https://github.com/google-ai-edge/gallery/blob/main/Android/src/app/src/main/java/com/google/ai/edge/gallery/customtasks/mobileactions/MobileActionsScreen.kt#L413 https://github.com/google-ai-edge/gallery/blob/main/Android/...
  - kaliqt 7mo ago
    That’s not the same app as posted which is for iOS.
    - selfsigned 7mo ago
      You're right, they reference IOS in the README.md and link to the GitHub repo from the Apple Store page yet don't include the IOS source, sneaky.
- kaliqt 7mo ago
  That is the Android repo, where is the iOS repo?

Sharmaji000 7mo ago

Still didnt release training recipe, data, methodology etc unlike deepseek. Mostly released to get developer ecosystem across their android built in ai. Still good and interesting, but not exactly philanthropic to the open source progress.

nickvec 7mo ago

Extremely impressed by how fast responses are on iPhone 17 Pro Max. Can’t wait for this to be used for Siri’s brain one of these days (hopefully!)

al_borland 7mo ago

I find it odd they are using the term “edge” to brand this, if it’s target is the general public. I’ve been to a few tech conferences and saw the term used there for the first time. It took me a little bit to see the pattern and understand what it meant. I have never heard the term used outside of those circles. It seems like “local” would be the term average users would be familiar with. Normal people don’t call their stuff “edge devices”.

deleted 7mo ago
[deleted]
bigyabai 7mo ago
> if it’s target is the general public. It's not - Apple is working with Google right now to make Siri into the public-facing version of this. This is kinda just the tech preview before all the branding has been painted on.
ycombinete 7mo ago
Funnily enough I work in the security industry and the term is ubiquitous there, so I didn’t even notice it.

danielrmay 7mo ago

I spent some time getting Gemma4-e4b working via llamacpp on iPhone and I'm really impressed so far! I posted a short video of an example application on LinkedIn here https://www.linkedin.com/feed/update/urn:li:activity:7446746072569491456/ https://www.linkedin.com/feed/update/urn:li:activity:7446746... (or x: https://x.com/danielrmay/status/2040971117419192553 https://x.com/danielrmay/status/2040971117419192553)

MysticOracle 7mo ago

Crashes for me on a couple of different iDevices (2 generations behind) after only a few 2-3 chats. Probably not enough RAM. Saw this one on X the other day updated with Gemma 4 and they have the built-in Apple Foundation model, Qwen3.5, and other models: Locally AI - https://locallyai.app/ https://locallyai.app/

LeonTing1010 7mo ago

[flagged]

rock_artist 7mo ago

I really believe in the future of local models. From app developer and user, My main concern for now is bloating devices. Until we’ll have something like Apples foundation model where multiple apps could share the same model it means we have something horrible as Electron in the sense, every app is a fully blown model (browser in the electron story) instead of reusing the model. With desktops we have DLL hell for years. But with sandboxed apps on mobile devices it becomes a bigger issue that I guess will/should be addressed by the OS. For my app I’ve been trying to add some logic based on large model but for bloating a simple Swift app with 2-3GB of model or even few hundred MBs feels wrong doing and conflicting with code reusability concepts.

janandonly 7mo ago
This app unlocks using the Apple Foundation model itself: https://apps.apple.com/nl/app/locally-ai-local-ai-chat/id6741426692 https://apps.apple.com/nl/app/locally-ai-local-ai-chat/id674...
hypersolo 7mo ago
[flagged]

modeless 7mo ago

It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model. Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.

ulfw 7mo ago
It runs very fast on my Qualcomm Elite Gen 5 SoC Oppo Find N6
- allpratik 7mo ago
  How many tokens per second? Also, does it get warm/hot?
  - modeless 7mo ago
    If this Gemma tokenizer I found online is accurate then my Pixel 10 Pro XL is getting ~22 tok/s on Gemma 4 E2B using the NPU, vs. 40 tok/s is what people are saying the MLX version gets on iPhone. Actually I found official performance numbers from Google saying iPhone gets 56 tok/s and Qualcomm gets 52. They don't even bother listing Tensor in their table. Maybe because it would be too embarrassing. Ouch! https://ai.google.dev/edge/litert-lm/overview https://ai.google.dev/edge/litert-lm/overview

jdthedisciple 7mo ago

it's Google, so is it really private? remember, megacorps are dying for infinite amounts of analytics data

classified 7mo ago
When I saw it wants me to "agree" to Google's "privacy policy", I deleted the app on the spot.
prism56 7mo ago
Are there any alternatives for on device android llm that aren't google and/or more private?

rcarmo 7mo ago

This is fun. I just wish I could add more skills, the UX is too dumbed down but knowing there is a run_js tool there is a lot that can be done here.

_nagu_ 7mo ago

If this works smoothly on iPhone, it could change how we think about mobile apps. Less backend dependency, more on-device intelligence.

jcutrell 7mo ago
This is what Apple promised a long time ago, and just couldn't quite connect on delivery.

totetsu 7mo ago

I have been looking at ARGmax https://www.argmaxinc.com/#SDK https://www.argmaxinc.com/#SDK for running on apple devices, but not sure yet at whats involved in porting a model to work with their sdk

gdzie-jest-sol 7mo ago

I need normal server too in local network I can run chat in other device and 'counting' on iphone. Second idea is input audio in other language, like Czech, Polish, French

orf 7mo ago

I’d recommend locally.ai[1] - it’s really good and has a wide range of models. Also has shortcuts support. 1. https://apps.apple.com/gb/app/locally-ai-local-ai-chat/id6741426692 https://apps.apple.com/gb/app/locally-ai-local-ai-chat/id674...

janandonly 7mo ago
Thanks for the link. Gemma 4 also works in this app.

imadselka 7mo ago

good model!

nightrate_ai 7mo ago

[dead]

MagicMoonlight 7mo ago

It seems really capable. A few more iterations of this and you won’t even need a subscription. All it needs is web search so that it can get up to date information.

derwiki 7mo ago

I asked it about the “Altamont Free Concert” (exact name of Wikipedia article), and it’s been a while since I’ve seen an hallucination this rich. Doesn’t give me confidence to use it.

Sukhesh-QA 7mo ago

[dead]

areys 7mo ago

[flagged]

haizhung 7mo ago

I encourage everybody to try this, if they have an iPhone. If you’re like me and don’t have the time to tinker with the latest and greatest all the time; this app lowers the barrier to entry significantly and provides a glimpse into what’s possible locally, on device. Honestly, I was extremely impressed by the speed and quality of the answers considering this thing runs on a phone. It honestly makes me want to sit down and spin up my own homegrown AI setup to go fully independent. Crazy.

ValveFan6969 7mo ago

[dead]

amai 7mo ago

The cooperation of Apple and Google is going to crush the competition: https://blog.google/company-news/inside-google/company-announcements/joint-statement-google-apple/ https://blog.google/company-news/inside-google/company-annou... The combination of Apples hardware and Googles software is unbeatable.

bigyabai 7mo ago
With Google's graveyard and Apple's walled garden, nothing can stop the enshittification train from trundling down the tracks.
lemonish97 7mo ago
Isn't their competition (at least in the mobile space) google/android themselves?

sshrajesh 7mo ago

> Note: I tried to hook this one up to OpenClaw and ran into issues Anyone worked on hooking up OpenClaw to gemma4 running locally?

areys 7mo ago

[flagged]

lemonish97 7mo ago

I hope they add a web search tool to the agent skills too. Most of my llm usage on my phone are just quick lookups and search summarizations. Would love to do these with a local model rather than Google AI mode of any other cloud based inference tools.

inzlab 7mo ago

Impressive

pseudosavant 7mo ago

It'd be fun to explore creating a Gemma 4 LLM API server app so you could use your iPhone's processing for agentic coding on a laptop. I don't know how useful it would be, but it'd be fun.

linncharm 7mo ago

[dead]

aimemobe 6mo ago

[flagged]

deleted 6mo ago

[deleted]

znnajdla 6mo ago

One really good use case for me is good, fast, offline translation. Both Apple Translate and Google Translate are worse in quality than a decent LLM and don’t work well offline. Gemma 4 is surprisingly good and often faster than waiting for an API call.