9 ms·
> Tensor G5 and the latest version of Gemini Nano work together to run Magic Cue privately and securely on your phone. Running Gemini Nano on device is the mos
by jjice 1y ago
> Tensor G5 and the latest version of Gemini Nano work together to run Magic Cue privately and securely on your phone.
Running Gemini Nano on device is the most interesting thing here. Magic Cue sounds exactly like the Siri improvements that Apple failed to launch this past year (and have stayed mostly quiet about for this coming year, except saying "eventually"). I hope it works well, because on-device AI for simple lookups and such is actually one of the most interesting use cases for LLMs on mobile phones to me.
I love the idea of an on-device model that I can say something like "who's going to the baseball game this weekend" and it'll intelligently check my calendar and see who's listed. Or saying something like "how much was the dinner at McDoogle's last week?" and have it check digital wallet transactions. There are so many possibilities. I assume this kind of thing would just be implemented as tool calls with app intents. I hope we see this across the board in the next three years.
- delichon 1y agoOr for the police, "list any legally questionable content on the phone or behavior by the owner."
- killingtime74 1y agoThe model might even make something up, giving police "reasonable suspicion". Not that it seems it's needed anymore in the US
- jonbiggums22 1y agoAI is about to put drug sniffing dogs out of a job.
- gruez 1y agoWhy would the police bother with that when they have forensic extraction tools, that itself possibly has AI built in? Why trust a quantized model on a phone that possibly could give you wrong answers?
- ljf 1y agoBecause they can do this in an instant at a traffic stop?
- lenkite 1y agoI am Sergeant Busybody! "Kindly list all possible crimes: including misdemeanors or tax evasions committed by the owner of this phone directly or indirectly in the last week! Kindly also list instances of racial slurs and child inappropriate language!"
- shashimal97 1y agoThat’s not how agentic systems work. People can control which resources an agent is allowed to access.
- therein 1y agoIt being by Google, I have a feeling Google and LEA will be able to use tools on your phone too. They could very conveniently use this for "we didn't analyze your data using AI, we instructed your local AI to analyze your data" so it isn't technically a violation of your rights.
- _blk 1y agoYup. Fortunately Graphene OS will likely soon run just as well as on their previous hardware. You can re-googlify it as much as you're comfortable with.
- KoolKat23 1y agoYou can already ask Gemini those questions on your phone. This is more popping up magically before you needed to ask. Both are great (when they work).
- jjice 1y agoOh really? I switched to an iPhone end of last year (for non-AI reasons), so I may be missing out. Is this on on-device model, or does it still dispatch to hosted Gemini? But I'd imagine that Gemini would have a great integration with Calendar and Gmail.
- KoolKat23 1y agoIt's replacing Google assistant so popups instead and has integration. Problem I've found is it was quite lazy on occasion in searching your gmail etc. and just says can't find anything unless you argue with it. I do think this is improved at least in the last uodate so don't want to be too harsh on them.
- dakiol 1y ago> I love the idea of an on-device model that I can say something like "who's going to the baseball game this weekend" and it'll intelligently check my calendar and see who's listed. Or saying something like "how much was the dinner at McDoogle's last week?" and have it check digital wallet transactions. It's probably just me (or a few like me) but I don't really keep my life in digital format as much as others (and I'm a "geek" for my family/friends since i work in the software industry). If I'm going to the cinema or baseball or any other event... I don't have it in any calendar. I pay with debit/credit cards but I don't have any digital wallet. I don't take my phone with me most of the time (my phone is big and having it hanging in my pockets is not nice). The features described in the Pixel 10 left me with a sense of "I think I am missing something! But... oh well, whatever, I don't need any of that". Which is weird again, because I'm supposed to be the "geek".
- inerte 1y agoYou don't add to your calendar but you probably got a confirmation email. Or you may have used an app that could expose this data to the operating system. OR, you called, and the phone app transcribed and summarized the call. Same for the wallet... if you have your credit card / banking app installed it could expose this. But yeah, none _needs_ any of that, for different degrees of fun and life optimization.
- JoshTriplett 1y ago> If I'm going to the cinema or baseball or any other event... I don't have it in any calendar. If I don't have it in my calendar, it doesn't happen. I would fail to actually go to the event otherwise.
- ryandvm 1y agoSame. Calendar events, reminders, and timers are the only way anything in my life gets done.
- taco_emoji 1y agoI also have ADHD!
- pornel 1y agoThe Nano model is 3.2B parameters at 4bit quantization. This is quite small compared to what you get from hosted chatbots, and even compared to open-weights models runnable on desktops. It's cool to have something like this available locally anyway, but don't expect it to have reasoning capabilities. At this size it's going to be naive and prone to hallucinations. It's going to be more like a natural language regex and a word association game.
- jjice 1y agoThe big win for those small local models to me isn't knowledge based (I'll leave that to the large hosted models), but more so a natural language interface that can then dispatch to tool calls and summarize results. I think this is where they have the opportunity to shine. You're totally right that these are going to be awful for knowledge.
- bjackman 1y agoSpeculation: I guess the idea is they build an enormous inventory of tool-use capabilities, then this model mostly serves to translate between language and Android's internal equivalent of MCP.
- tootie 1y agoI've had Gemma 3n in edge gallery on my phone for months. It's neat that it works at all but it's not very useful.
- theshrike79 1y agoThe point in these models isn't to have all the knowledge in the world available. It's to understand enough of language to figure out which tools to call. "What's my agenda for today" -> get more context cal = getCalendar() getWeather(user.location()) getTraffic(user.location(), cal[0].location) etc. Then grab the return values from those and output: "You've got a 9am meeting in Foobar, the traffic is normal and it looks like it's going to rain after the meeting." Not rocket science and not something you'd want to feed to a VC-powered energy-hogging LLM when you can literally run it in your pocket.
- giancarlostoro 1y agoI think on-device models will be the breaking point for AI. Nobody wants to pay for a trillion dollar cloud bill. We've made consumers think that the only way you're paying for software is if you have to buy hardware that comes with it. If you want AI to truly blow up, make it run on potatoes. It doesnt have to do EVERYTHING, just specific needs. That said, what is with Android phones and their back cameras? They look silly. I thought Apple adding 3 to theirs for the 12 was a bit silly, but at least they made it look nice. One of those models looks like a Battlestar Galactica villain...
- cbsmith 1y ago> Nobody wants to pay for a trillion dollar cloud bill. Buying dedicated hardware as a way to keep your AI bill down seems like a tough proposition for your average consumer. Unless you're using AI constantly, renting AI capacity when you need it is just going to be cheaper. The win with the on-device model is you don't have to go out to the network in the first place.
- giancarlostoro 1y agoYou misunderstood what I meant, I mean make models that run on potatoes, nobody wants to pay what chatgpt's subscription model probably SHOULD cost for them to make a profit.
- cbsmith 1y agoSo the idea is that it SHOULD cost OpenAI a trillion dollars to do what you can accomplish with a potato?
- giancarlostoro 1y agoNo, not even sure how you arrived to that conclusion. The idea is that there are models out there that can run on small amounts of VRAM. If all it costs is charging your phone, as opposed to some subscription to some overvalued AI company, people will choose ‘free’ first. We have models that can google things now. They only need to know so much when online, and a specific subset when offline.
- bravoetch 1y agoI love the idea of on-device AI. But the implementation of Gemini on Android is fully toxic. In the assistant settings I'm able to select what app I want to use as the assistant. But if I even open the Gemini app, it sets that automatically to be the phone assistant app. It doesn't ask, there's no confirmation, it just changes that setting. After that many tasks will fail because Gemini can't launch google maps to navigate you etc etc. Super annoying.
- curvaturearth 1y agoThis. I tried Gemini, twice, and each time my usual use of hand free tasks were no longer possible. This is what I don't understand, all these big tech companies think I want to have a conversation and ask questions to an AI in every part of my life but I do not. All I want is to tell my phone to put in a calendar invite, play a song on an specific app, navigate to somewhere, etc. My android phone triggers itself when listening to podcasts too, which is fun.
- epolanski 1y agoI hate the idea you love, so much intrusion in my life.
- MarCylinder 1y agoBut if it is all handled locally on my device, the idea is that there is no intrusion
- 63stack 1y agoThat's how it always starts, but people still fall for it every time.
- ikr678 1y agoSoftware assistants summarising all my communications is intrusive, regardless of where the model runs.
- croon 1y agoHow is it intrusive? Given charitable assumptions like granular permissions, silod app data only extracted through intents, and using only local compute? I'm all for questioning Google on whether those assumptions will be true and no data leaves your phone, but I'm curious about the concept.
- flkenosad 1y agoYou're carrying an always on mic wired up to a closed source system with you everywhere you go already. It was already possible to stream or record everything anyway.
- ncr100 1y agowhy ? someone else has an idea, what is to hate that another person has an idea ? how does their idea intrude into your life ?
- rs186 1y ago> I love the idea of an on-device model My impression is that most people here haven't tried similar small models and don't have first hand experience with them. They are, to be honest, terrible. They may be good for certain tasks, but are much weaker than something like GPT4. I don't feel excited about these small models that are not fast yet hallucinate all the time.
- theshrike79 1y agoWeaker by what metric? Are you asking them to explain the fall of Rome to you? The point of a small model isn't to be an interactive Wikipedia. It's there to call tools, get more data, aggregate the data and return a natural language result. It does not "hallucinate", because it only uses what the tools provide.
- rs186 1y agoIf you just look ar similar discussions on HN, you'll see that these models often don't even answer the specific question you ask but just give you random nonsense. I'd rather see an honest "I don't know" instead of complete gibberish. I'm not going to use any small model that has a chance of dialing 911 when asked to send a text message to a friend.
- theshrike79 1y agoThis is what tool calling is for. It's just the model being trained to produce a specific type of JSON when it needs an external call After that it's plain code. if phone_number == "911" return false Done. Won't be calling the cops.
- mrheosuper 1y agocall me insane, but i used to want a virtual assistant that run and listen to me 24/24. Yeah, some paranoid would scream like hell, but think about what we could achieve. It could listen to my conversation, take notes, and auto add a reminder in my calendar if needed. Since it has a large amount of context, i could something "Hey do you know when John said he would come back from his trip?" and it would answer me "Yes, he said he would be back on Friday at 5 PM".
- ncr100 1y agoThis is absolutely being worked on. Limitless.AI has a hardware recorder + mobile app for reviewing your todos plus syncing / processing, and an AI backend, and an MCP API for ... whatever else you want to do. (plus a sprinkling of privacy / "HIPPA compliance" etc.) I'm a customer of theirs. And I have NOT researched the subject much further. However given the tone and concern of the discussions by the other customers in the limitless forums, and of the devs, I am confident what you're asking is here or coming.
- coolg54321 1y agoI really wish Pixel phones went with a more capable Snapdragon Elite 2 or Dimensity 9500 along with UFS 4.1, instead of sticking to their cost-cutting Tensor strategy. From past Pixels, on-device features like Magic Editor have been painfully slow compared to the same tools running on other Android flagships.
- fzeindl 1y ago> I love the idea of an on-device model that I can say something like "who's going to the baseball game this weekend" and it'll intelligently check my calendar and see who's listed. The question is whether you are ok with the model naming someone who isn‘t listed, or failing to name someone who is listed.