6 ms·
Show HN: Off Grid – Run AI text, image gen, vision offline on your phone
Your phone has a GPU more powerful than most 2018 laptops. Right now it sits idle while you pay monthly subscriptions to run AI on someone else's server, sending your conversations, your photos, your voice to companies whose privacy policy you've never read. Off Grid is an open-source app that puts that hardware to work. Text generation, image generation, vision AI, voice transcription — all running on your phone, all offline, nothing ever uploaded.
That means you can use AI on a flight with no wifi. In a country with internet censorship. In a hospital where cloud services are a compliance nightmare. Or just because you'd rather not have your journal entries sitting in someone's training data.
The tech: llama.cpp for text (15-30 tok/s, any GGUF model), Stable Diffusion for images (5-10s on Snapdragon NPU), Whisper for voice, SmolVLM/Qwen3-VL for vision. Hardware-accelerated on both Android (QNN, OpenCL) and iOS (Core ML, ANE, Metal).
MIT licensed. Android APK on GitHub Releases. Build from source for iOS.
- bkmeneguello 8mo agoVery nice, but I'm gonna wait for the f-droid build.
- ali_chherawalla 8mo agook, let me figure that one out.
- saratogacx 8mo agoI've found using Obtanium to be a great tool for tracking github android apps that aren't listed on repos like f-droid
- derac 8mo agoI haven't run it, but I looked through the repo. It looks very well thought out, the UI is nice. I appreciate the ethos behind the local/offline design. Cheers.
- ali_chherawalla 8mo agothank you!
- snicky 8mo agoGitHub Releases link is broken. The dash in "off-grid" is missing.
- ali_chherawalla 8mo agoyup, just took a look at that and fixed it. My bad!
- flyingkiwi44 8mo agoThe repository is listed as offgrid-mobile everywhere on that page but is off-grid-mobile. So the lastest releases is at https://github.com/alichherawalla/off-grid-mobile/releases/latest https://github.com/alichherawalla/off-grid-mobile/releases/l... And the clone would be: git clone https://github.com/alichherawalla/off-grid-mobile.git https://github.com/alichherawalla/off-grid-mobile.git
- ali_chherawalla 8mo agohey, yes. thanks just pushed that fix out
- flyingkiwi44 8mo agoLooks like the build requirements for SDK and NDK in the build instructions don't match the build.gradle. -Android SDK (API 34) -Android NDK r26 vs compileSdkVersion = 36 targetSdkVersion = 36 ndkVersion = "27.1.12297006"
- ali_chherawalla 8mo agoyup just updated. Thanks!
- vachina 8mo agoOk it lists the instruction to build for iOS, but how to sideload?
- behole 8mo agoI found more info https://github.com/alichherawalla/off-grid-mobile/blob/main/docs/ARCHITECTURE.md#building-from-source https://github.com/alichherawalla/off-grid-mobile/blob/main/...
- ali_chherawalla 8mo agoyeah thats right. for now for iOS you'll actually have to pod install etc. Thanks for pointing this out
- instagib 8mo agoWonder if you can use GitHub actions to build iOS. I found a guide for virtual box macOS which failed on intel then another for hyper-V but haven’t tried that one yet.
- ali_chherawalla 8mo agoOff Grid is now live on the App Store: https://apps.apple.com/us/app/off-grid-local-ai/id6759299882 https://apps.apple.com/us/app/off-grid-local-ai/id6759299882
- resonious 8mo agoOn my Samsung phone it doesn't move the screen up to make room for the keyboard so I can't see what I'm typing. Really awesome idea though. I want this to work.
- dotancohen 8mo agoI can confirm this bug on a Samsung S24 Ultra.
- ali_chherawalla 8mo agosorry about that one. I'm taking a look and fixing it right away
- ali_chherawalla 8mo agohey, just pushed a fix for it here: https://github.com/alichherawalla/off-grid-mobile/releases/tag/v0.0.45 https://github.com/alichherawalla/off-grid-mobile/releases/t... Thanks for spotting and reporting this.
- dotancohen 8mo agoGreat work! I can confirm that the issue is resolved on the Samsung S24 Ultra.
- ali_chherawalla 8mo agoawesome. thanks!
- sangaya 8mo agoPutting the power and the data of the users in the hands of the users themselves! Well done. Getting it setup was easy. Wish the app recognized the keyboard and realized when it was displayed so the bottom menu and chat box weren't hidden under it.
- ali_chherawalla 8mo agoThank you!
- nine_k 8mo agoIs there something similar, but geared towards a Linux desktop / laptop? I suppose this would be relatively easy to adapt.
- ali_chherawalla 8mo agoLM Studio solves for it pretty well I think. It doesn't do image gen etc though
- jeroenhd 8mo agoIf you have enough (V)RAM you can set up Ollama + any of the generation tools here (https://docs.openwebui.com/category/create--edit-images https://docs.openwebui.com/category/create--edit-images) and hook them into OpenWebUI to get something pretty close to what you'd get on a cloud AI chatbox. Decently performing models for day to day use cases use at least 6GB VRAM each, though, and even then they're not coming very close to what the cheapest AI websites offer.
- chr15m 8mo agoThis rules. Godspeed!
- ali_chherawalla 8mo agowow. thank you
- wittlesus 8mo ago[dead]
- add-sub-mul-div 8mo ago> that's basically usable for real conversations. That's using the word "real" very loosely.
- ali_chherawalla 8mo agoI've added a section for recommended models. So basically you can chose from there. I'd recommend going for any quantized 1B parameter model. So you can look at llama 3.2 1B, gemma3 1B, qwen3 VL 2B (if you'd like vision) Appreciate the kind words!
- durhamg 8mo agoThis sounds exactly like Claude wrote it. I've noticed Claude saying "genuinely" a lot lately, and the "real killer feature" segue just feels like Claude being asked to review something.
- Terretta 8mo ago> The fact that you're getting 15-30 tok/s for text gen on phone hardware is wild — that's basically usable for real conversations. Wild how bad it is compared to, say, Russet for iOS/ipadOS, which runs these same models at 110 tps.
- noodletheworld 8mo agoNice idea, but isnt this kind of daft? There are basically no useful models that run on phone hardware. > Results vary by model size and quantization. I bet they do. Look, if you cant run models on your desktop, theres no way in hell they run on your phone. The problem with all of these self hosting solutions is that the actual models you can run on them aren't any good. Not like, “chat gpt a year ago” not good. Like, “its a potato pop pop” no good. Unsloth has a good guide on running qwen3 (1), and the tldr is basically, its not really good unless you run a big version. The iphone 17 pro has 12GB of ram. That is, to be fair, enough to run some small stable diffusion models, but it isnt enough to run run a decent quant of qwen3. You need about 64 GB for that. So… i dunno. This feels like a bunch of empty promises; yes, technically it can run some models, but how useful is it actually? Self hosting needs next gen hardware. This gen of desktop hardware isnt good enough, even remotely, to compare to server api options. Running on mobile devices is probably still a way away. (1) - https://unsloth.ai/docs/models/qwen3-how-to-run-and-fine-tune https://unsloth.ai/docs/models/qwen3-how-to-run-and-fine-tun...
- iddan 8mo agoI think if people will people know how accessible it is to run local LLMs on their device they will consider buying devices with more memory that will be able to run better models. Local LLMs in the long run are game changers
- ali_chherawalla 8mo agoI agree. I mean mobile devices have only been getting more and more powerful.
- deleted 8mo ago[deleted]
- resonious 8mo agoThe app is basically just a wrapper that makes it super easy to set this up, which I'm very thankful for. I sometimes want to toy with this stuff but the amount of tinkering and gluing things together needed to just get a chat going is always too much for me. The fact that the quality of the AI isn't good is just the models not being quite there yet. If the models get better, this app will be killer. If there's a similar app for desktop that can set up the stronger models for me, I'd love to hear about it.
- imposter 8mo ago[dead]
- numpad0 8mo agoHasn't here been few of those already eg [1][2]? 1: https://github.com/a-ghorbani/pocketpal-ai https://github.com/a-ghorbani/pocketpal-ai 2: https://github.com/shubham0204/SmolChat-Android https://github.com/shubham0204/SmolChat-Android
- ali_chherawalla 8mo agoi don't think they do image gen etc? You can almost think of Off Grid like an on-device assistant. It'll handle everything like vision, and attachments too. We're still early stages and lots of optimizations to do, but we'll get there.
- ucguy4u 8mo agotried this on pixel 9, interesting stuff
- ali_chherawalla 8mo agoawesome
- ruthlesssavage 8mo agoGame-changer.Just downloaded it and wow,this is the future. It’s incredible to have this much power fully offline. No lag, no privacy concerns, just pure utility. I’ve been playing with the vision AI all morning and it hasn't crashed once. 10/10. I have also used other GitHub projects but this feels premium, faster and optimised. Thank you for this!
- ali_chherawalla 8mo agothank you, appreciate it!
- romperstomper 8mo agoAre there any nice TTS models? It could be handy to have them as well.
- ali_chherawalla 8mo agoI don't really do TTS right now, but i allow for STT using whisper. TTS is part of the backlog and I should be adding it soon
- willx86 8mo agoCheck out "FUTO" keyboard Best on device STT I've ever used
- ali_chherawalla 8mo agosure, I'll take a look at it. Thank you for the recommendation
- jeroenhd 8mo agoLooks useful, though something went wrong doing NPU image generation on my phone. Reminds me a lot of https://github.com/google-ai-edge/gallery https://github.com/google-ai-edge/gallery which is a proof-of-concept app by Google themselves for their AI libraries. However, your app supports more and larger models without having to manually import anything, which is very useful.
- ali_chherawalla 8mo agoi'm adding another feature to recommend models based on h/w. that should take care of this stuff
- willx86 8mo agoI like it, I have a couple minor things (android) Markdown rendering would be great Letting me download a model, wait for it, then. tell me I can't run it safely in my ram feels like it could've told me at the beginning Bar that nice! Was it annoying integrating with apples AI stuff?
- ali_chherawalla 8mo agohey! thank you for the feedback. appreciate it. I pushed something recently that actually prevents showing up models in the download screen that don't fit on your phone. Please give it a whirl. Yeah it wasn't straight forward figuring out the speed + getting it to work for both Android and iOS. Fair bit if complexity. But I'm so happy I got it done.
- isawczuk 8mo agoI was looking for something like this. Are you open for contributions?
- ali_chherawalla 8mo agoawesome. The more the merrier. Feel free to get started. I'm still working on a few things, but let me get a backlog in place and push it so people know what the roadmap looks like
- Swauzie 8mo agoI gave it a try on Graphene OS: I downloaded models 1st, then blocked Network Accesses. The Apps works fine for text generation, but not for image generation: it needs network access... I did not search where it needs to go.
- ali_chherawalla 8mo agoit doesn't need internet to generate an image. it needs it to download the model, and to be fair if you've got the zip for the model already you can just import that
- Swauzie 8mo agoI have download the model... only then I have disabled network access. It loops with an error message needing access to DSP ? Anyway, I like what you did, as I do not want to train AI with my inputs this is perfect.
- ali_chherawalla 8mo agoI think thats a bug, I'm guessing you're trying one of the NPU models? If you drop to CPU for now it should hold. Only phones with qualcomm chips are able to use the NPU. I'm working towards changing that. but yeah just to be clear there is no internet needed to run any of this. Infact I'm so averse to it, I've not even added analytics for this one. So flying pretty blind here.
- ali_chherawalla 8mo agoOff Grid is live on the playstore now. Check it out: https://play.google.com/store/apps/details?id=ai.offgridmobile https://play.google.com/store/apps/details?id=ai.offgridmobi...