9 ms·
Gemini Live with camera and screen sharing capabilities
- polishdude20 2y agoWhy only Pixel 9? Surely none of the computation is on-device anyways
- deleted 2y ago[deleted]
- ipsum2 2y agoThe second sentence contradicts your statement This soon will be available to all Gemini Advanced subscribers on Android devices, and today we’re bringing it to more people, starting with all Gemini app users on Pixel 9 and Samsung Galaxy S25 devices
- bytesandbits 2y agocurious, what is done on-device?
- echelon 2y agoLikely nothing. Mobile devices can't do much AI stuff except for the tiniest of models, and it'll likely be a long time before they will be able to anything super useful. Edge compute is a long ways off, even on desktop without a dedicated GPU. But especially on mobile. You might have bumped into a media website trying to run a WASM-powered onnx runtime background removal tool, or perhaps a super slim LLM. You'll notice how slow these are and how they can lock up your browser. That's about the experience you can expect from edge compute. Nvidia's proclamation that they're going to be working on robotics as their next growth sector could mean more innovation on the edge / low power compute front. But most of the yield will come from better model architectures and models designed specifically to work with compute constraints. For now, datacenter inference reigns supreme.
- pzo 2y ago> WASM-powered onnx runtime background removal tool Yes it's slow because they will most likely execute on CPU (for sure on iOS). WebGPU is still not enabled on safari and WebNN is not even supported anywhere (so that you can use NPU provider). ONNXRuntime is not even the most optimised when running natively (instead of WASM) on iOS, e.g. doesn't support MPS provider (GPU) and NPU provider (via CoreML) implements only subset operators (last time I tried). Safari also provide limitation on WASM memory usage In practice when you want the best performance you would have to use native app and CoreML with NPU provider and model architecture optimized for NPU. On iOS for now big limitation is available RAM, but even my iPhone 13 mini has exactly the same fast NPU as on my Macbook M2 Max when tested it's having similar speed to running on GPU.
- deleted 2y ago[deleted]
- addandsubtract 2y agoI still can't use the "summarize" feature that was used in marketing to sell the Pixel 8 Pro. Not holding my breath on ever being able to use this, either.
- cavisne 2y agoYou can do the same from the browser on any phone (including iphone) by going through AI Studio https://aistudio.google.com/live https://aistudio.google.com/live
- clueless 2y agoUpvoted. For a couple of weeks now, I added that site as a bookmarked app on my iPhone just to use that Gemini live feature, though I can only use the camera/voice feature (no screen sharing ability on the chrome ios browser afaik). Super weird they are not releasing this feature more officially on iOS
- colecut 2y ago"The Google Pixel 9 is powered by the Tensor G4 chip, a custom-designed processor optimized for AI. This chip, along with 12 GB of RAM (or 16 GB for Pro models), enables advanced AI features and enhances overall performance."
- cavisne 2y agoNone of that onboard stuff ended up working afaik as the Tensor chips shipped right as giant LLM's became how dominant.
- 0x1ceb00da 2y agoIncentives to buy new hardware.
- sureglymop 2y agoWatching that demo video, I wonder why they chose to use that? Gemini only talked about some useless surface knowledge that would be forgotten quickly, whereas if she actually read the Wikipedia page she would learn more and retain it better.
- clueless 2y agoright? not great examples. Here are the kind of prompt examples I would have used instead: - explain this symbol to me - what TV show am I watching - how much is the house I'm looking at worth (bring in geo-location) - How often should I be watering this plant - how many calories are in this meal - what type of car is this - whats the exact name/type of this screw part - what are the exact dimensions of this door frame - what kind of style of art is this - how often should I be taking this medication - how many copies has this book sold - which part of the world is this pic from
- dimitri-vs 2y agoBecause besides the TV show one none of your examples benefit from the video/"live" modality? Or at least I wouldn't be motivated to use video for that.
- com2kid 2y agoMy meta glasses can do a decent number of those, with the ideal form factor for those use cases.
- chem83 2y agoLooks like Google was demonstrating some of that on stage recently: https://gizmodo.com/google-showed-off-advanced-ar-glasses-that-may-never-leave-the-lab-2000587572 https://gizmodo.com/google-showed-off-advanced-ar-glasses-th...
- kristopolous 2y agoor even the internet one. It would have been nice if it was like "I see you have a NetMan 402-BX with a blinking amber light. That's a common failure mode that can be fixed by updating the firmware. You need to press and hold the power button and put a pin into the reset hole..." You know ... something more than "reboot that shit"
- tintor 2y ago"Screen images simulated."
- sksxihve 2y ago[flagged]
- deleted 2y ago[deleted]
- dimitri-vs 2y agoDon't be so pessimistic, how about these uses: - Let Gemini watch you drive so you can get advice on driving better - Let Gemini help you rewire your outlet for the first time - Let Gemini help you pick edible mushrooms in the forest
- jmugan 2y agoIt's funny, you are kidding, with the impending death and such, but having it be able to look at what you are seeing seems useful to me. E.g., "How do I work this contraption?"
- devsda 2y ago> Don't be so pessimistic, how about these uses: I suspect, these examples are chosen specifically because their failures modes can be lethal ?
- kibibu 2y agoHow about: - Gemini, help me tell if this is sarcasm
- qingcharles 2y ago"Gemini, what's the best way to bury this body?" "Gemini, please count the cockroaches in this kitchen." "Gemini, calculate my survival chances sprinting across this interstate." "Gemini, speculate on the percentage volume of pee-pee in this swimming pool." "Gemini, how many days will this meal set back my life expectancy?" "Gemini, will these socks lead to my arrest by the fashion police?" "Gemini, my cat is making that face again -- has it found out it's adopted?"
- 2y ago
- deleted 2y ago[deleted]
- vorpalhex 2y agoIs anyone seriously using Gemini daily? How is it compared to other agents you've tried? Do you feel it's a good value prop? What does it excel/fail at?
- FactolSarin 2y agoI already pay for Google One storage, so it's the cheapest of the paid LLMs for me. That's 99% of the reason I use it, and honestly I don't really have any strong opinions on it compared to ChatGPT. It's about the same level, and with new models constantly being released for all the different LLMs I've kind of lost track of what it's particularly good or bad at compared to others. I will say the video with Gemini live is pretty impressive. My family and I tried it a bit yesterday, and my kids wanted to show Gemini all our pets. My kid showed it our cat, picking it up roughly as she is wont to do, and I was impressed when it asked "Is [name of cat] always so patient being handled like that?"
- diggan 2y ago> I already pay for Google One storage, so it's the cheapest of the paid LLMs for me. That's 99% of the reason I use it, and honestly I don't really have any strong opinions on it compared to ChatGPT I'm on a ChatGPT pro plan, been using it for a good while but got an offer on Google One storage so tried it out for a month. Google's models are far behind compared to OpenAI's, and seemingly o1 Pro Mode is still the best out there, albeit slow obviously. But probably the model I've got furthest with on difficult problems, and even the "simpler" models from OpenAI are still better than Gemma 2.5. It does seem that Google has better tooling available for their models though, so a combination of the tooling of Google with the models of OpenAI would probably be optimal, but unlikely we'll see that happen.
- whimsicalism 2y agohow long ago? i think 2.5 exp is better than o1 or o3-mini in my experience.
- lytedev 2y ago
- kovek 2y agoI think it would be nice if the Pixel Fold could do: Have a browser on the left showing some content, and have Gemini on the right, where you can prompt it with questions or asking it to take actions on the left.
- discordance 2y agoand then humanity became illiterate
- behnamoh 2y agoThis is Apple Intelligence the way it was supposed to be ("AI for the rest of us"), but Apple just doesn't "get" AI, so here we are—the only platform provider that is taking the correct approach to AI is Google.
- Workaccount2 2y agoIt seems very likley that Apple will license this in some form. They already are testing with ChatGPT and Gemini, but I don't think OpenAI is going to be able to undercut Google on price. And how it looks right now they might not be able to beat them on performance either. Deep down though I can feel Apple must have some level of aversion to this. They have been trying to untangle themselves from Google for so long.
- cadamsdotcom 2y agoMarketing should use their imagination! Imagine putting dice and random objects (cups, forks..) on a table, pointing your phone at them and asking it to invent a new game for your friends. Tell it to use these objects and also use the live camera as a gameplay element. Or recognizing bird or plant species. Or helping a blind person go hiking, helping avoid tree roots and describing the beautiful scenes they’re in. So much possibility!
- wgd 2y agoHow charitable of you to assume those examples work reliably.
- jryle70 2y agoAre you willing to bet that it wouldn't work reliably in a year, 2 years, 5 years?
- pfg_ 2y agoIf you're releasing something today, should you talk about what it can do now or what it might be able to do in two years?
- jskarpet 2y agoBemyeyes app already work quite reliability to describe scenes to the blind
- lynx97 2y agoHaha, did you evaluate this personally? I did a BeMyEyes test recently, trying to sort about 40 cans according to the existance of a deposit logo. After 90 minutes of submitting photos, and a second round to make sure it doesn't lie too much, I had 16 cans which according to BeMyEyes (OpenAI) had a deposit logo. Then, I went to the shop to bring them back. Turns out, only 4 cans had a logo. So after a second round to eliminate hallucinations, the success rate was only 25%. Do you call that reliable?
- randomsofr 2y agopretty useless demos haha, i wonder why they choose those cases, maybe it really doesn't do much else correctly rn
- sandspar 2y agoCool! I'm looking forward to having a live AI drawing tutor. None of the models are there yet but we're getting close!
- devinprater 2y agoIt's not the best at helping me play video games yet, lol. Ah well. Blind people are used to waiting. :)
- simonw 2y agoI've been using this to help me read papers with mathematical notation in them. I screen share with https://aistudio.google.com/live https://aistudio.google.com/live and then ask (out loud) questions like "what do these symbols mean?" - it's a huge help for me, since I never memorized all of that stuff.
- rockmeamedee 2y agoOh this is great! I always have this problem. I find that's one of my biggest barriers when reading queueing theory content. I'm only doing it intermittently so I don't have memorized the meanings of ρ,σ,μ,λ... Visually I also often confuse rho and sigma, and math texts will use psi ψ and phi φ in weird fonts and I can never tell them apart.
- yalogin 2y agoI am getting extremely skeptical of AI “age”. I was hoping it will unlock a whole new generation of startups like the iPhone did. However genAI is too generic and too blunt a tool, as in it does everything. However it’s too expensive for a small company to do stuff with. Looks like these AI companies (google and OpenAI) realize that and so are even doing the vertical integration themselves. In such an event does genAI end up being the automation tool that you access/use through OpenAI or google and that’s it? I am sure people here see it better than I do, so what new class of problems is this genAI going to solve?
- dmos62 2y agoIt's not too expensive. I did some crawling the other day. AI wrote the crawler automatically, then I had it extract structured info from the crawled product info. It was about 600 requests to o4-mini. Cost me about 2.5 cents.
- occamschainsaw 2y agoo4-mini? It is a rumored model building up on o3, and different from 4o-mini. Sorry, their naming scheme is a mess. Would be cool if you do have o4-mini access though. https://www.theverge.com/news/646458/openai-gpt-4-1-ai-model https://www.theverge.com/news/646458/openai-gpt-4-1-ai-model
- dmos62 2y agoMy mistake, I meant GPT-4o mini.
- fhd2 2y agoWell, what's a tool? I would say: 1. It solves a problem. Doesn't have to be a completely unsolved problem, can just be a new solution. Or even just new packaging on an old solution. But it needs to solve some kind of problem. 2. It's trustworthy. Some people get a tool to suite their own process. But the majority, from anecdotal evidence, will adopt the tool's process. There's this idea that "these guys know how to do invoicing so I don't have to think about invoicing if I use their invoicing tool". 3. It's known. A bit philosophical, but if something exists that nobody _knows_ solves a problem they might not even know they have, how much of a useful tool is it, really? DropBox is an interesting example. It wasn't exactly a major scientific breakthrough, and a lot of people asked "why don't people just use FTP?". If you focus on (1), DropBox looked close to pointless. But what they did is nail (2) and (3). Now, if you subscribe to the hype, you might argue (1) and (2) will soon be covered. AI will magically solve your problem and be a universal domain expert telling you what to do, so you don't have to think about it. You might also argue that it will magically solve (3), with stuff like Gemini Live kinda watching you all day and constantly going "let me tell you how to do that" or "let me do that for you". Seems unlikely to me. Not impossible, most things I can think of are theoretically possible. Just unlikely. And if you think even just _one_ of those three aspects can't be fully automated in the near future, there's still plenty of opportunity left to differentiate in that area. I think generative AI does unlock a new generation of startups, because it's genuinely new technologies that we can find at least some valuable use cases for. And an army of startups tends to be better at quickly exploring a new solution space than a few big incumbents. So in that sense, it is similar to smartphones, which also brought a new solution space, and with it, startups.
- Timwi 2y agoI don't understand why Android only? Why would nobody want to use this on a PC?
- pcdoodle 2y agoNo thanks google.
- kleiba 2y agoCool tech, but for some reason, the very first sentence in every reply the AI gives in the demo videos is really off-putting to me personally. It seems to me like perhaps this kind of joyful helpfulness introductory sentence is expected in US culture, but it immediately turned me off. I come from a culture that is less verbose in general and more to the point, so this feels like a mismatch right away.
- 1024core 2y ago[flagged]
- d-lisp 2y agoThat is not what is implied here, OP seems to dislike the speech aesthetics produced by the model. I feel the same; the sugar-coating provided before and after any actual valuable information is (to me) : - not succeeding in awkwardly trying to achieve an experience that would be comparable to talking with a human person - not efficient, not enjoyable - perfectly matching the experience of talking with an highly standardized and gimmicky version of human_v0.01. Now, that being said, I don't really care about all of this. The USA population is equivalent to approximately 4% of the total world population.
- terminatornet 2y agothe first example video tells you how to improve your home decor by saying that you could add a side table and blanket to your chair. Thank you Gemini for telling me that tables can go next to chairs. There's really no use for AI outside of making studio ghibli drawings and giving me a bunch of broken code quickly.
- netsharc 2y agoThe answer seems to also be very surface-level, wow it sees a chair and lamp in a corner and believes it's the perfect position, without apparently considering anything else in the room. Marie Kondo doesn't need to worry about being replaced by AI, yet...
- 1024core 2y agoIs there a way to record the screen WITH AUDIO and save it?