6 ms·

https://aistudio.google.com/live https://aistudio.google.com/live is by far the coolest thing here. You can just go there and share your screen or camera and ha

https://aistudio.google.com/live https://aistudio.google.com/live is by far the coolest thing here. You can just go there and share your screen or camera and have a running live voice conversation with Gemini about anything you're looking at. As much as you want, for free. I just tried having it teach me how to use Blender. It seems like it could actually be super helpful for beginners, as it has decent knowledge of the toolbars and keyboard shortcuts and can give you advice based on what it sees you doing on your screen. It also watched me play Indiana Jones and the Great Circle, and it successfully identified some of the characters and told me some information about them. You can enable "Grounding" in the sidebar to let it use Google Search even in voice mode. The video streaming and integrated search make it far more useful than ChatGPT Advanced Voice mode is currently.

selvan 2y ago
Get started documentation on Multimodal Live API : https://ai.google.dev/api/multimodal-live https://ai.google.dev/api/multimodal-live
Brotkrumen 2y ago
Sounds interesting, but voice input isn't working for me there. I guess I'm too niche with my Mac and Firefox setup.
- mentalgear 2y ago
  Actually plenty of tech people are using mac & firefox
  - portaouflop 2y ago
    Irony detectors malfunctioning perhaps?
    - baq 2y ago
      'irony' meant 'something made of metal' last time I checked
      - socksy 2y ago
        Right, and Macs are made out of aluminium
        Alifatisk 2y ago
        What is Firefox made out of then?
        shakna 2y ago
        Aluminum. It's American.
        internet_points 2y ago
        The amount of rust indicates iron. So Firefox is very irony.
        cyanwave 2y ago
        Wood.
        dudeinjapan 2y ago
        Fire and foxes, presumably.
      - deleted 2y ago
        [deleted]
- littlestymaar 2y ago
  This isnt entierly suprising as Google have been breaking things artificially on Firefox for years now (Google Map and YouTube at least). Maybe try spoofing Chrome's user-agent.
- SkyPuncher 2y ago
  Console is throwing an error: "Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported." Quick research suggests this is part of Firefox's anti-fingerprinting functionality.
sky2224 2y ago
THIS is the thing I'm excited for with AI. I'm someone that becomes about 5x more productive when I have a person watching or just checking in on me (even if they're just hovering there). Having AI to basically be that "parent" to kick me into gear would be so helpful. 90% of the time my problems are because I need someone to help keep the gears turning for me, but there isn't always someone available. This has the potential to be a person that's always available.
- Huppie 2y ago
  Just as an FYI: I recently learned (here on HN) that this is called Body Doubling[0] there's some services around (there's at least one by someone that hangs around here) that can do this too. [0] https://en.m.wikipedia.org/wiki/Body_doubling https://en.m.wikipedia.org/wiki/Body_doubling
  - nolist_policy 2y ago
    Also, there are co-working spaces in VRChat, which works wonders for me. I went to the Glass Office co-working space to study for exams this summer and it worked out really well. I also met some nice people there. A standalone Quest 3 is enough to get you started.
  - themaninthedark 2y ago
    Do they support WFH setups?
    - mklepaczewski 2y ago
      The parent might be referring to us: https://workmode.net/ https://workmode.net/. Most of our clients work from home. Do you have a specific concern about body doubling and working from home?
      - kirubakaran 2y ago
        Some unsolicited feedback (please feel free to ignore): When I click on "Pricing" in the nav bar, it scrolls down, and the first thing that catches my eye is "$2100 / month". I happened to see this time that this is the benefit you're projecting and it is actually $2.50/hour. On the previous times I've visited your website based on your HN comments, I've always thought $2100/month was what you were going to charge me and closed the tab. I've been frustrated myself that people don't read what's right there on the page when they come to my startup app's landing page. Turns out I do the same. Hope this helps you improve the layout / font sizes and such "information hierarchy" so the correct information is conveyed at a glance. IMHO $2.50/hour is great value, and stands on its own. I know how much my time is worth, so perhaps the page doesn't really have to shout that to convince me. Again, please feel free to ignore this as it is quite possible that it is just me with the attention span of a goldfish with CTE while clicking around new websites.
internet_points 2y ago
I got so hopefuly by your comment I showed it my current bug that I'm working on, I even prepared everything first with my github issue, the relevant code, the terminal with the failing tests, and I pasted it the full contents of the file and explained carefully what I wanted to achieve. As I was doing this, it repeated back to me everything I said, saying things like "if I understand correctly you're showing me a file called foo dot see see pee" and "I see you have a github issue open called extraneous spaces in frobnicator issue number sixty six" and "I see you have shared some extensive code" and after some more of this "validation"-speak it started repeating the full contents of the file like "backquote backquote backquote forward slash star .. import ess tee dee colon colon .." and so on. Not quite up to excited junior-level programmer standards yet. But maybe good for other things who knows.
- nolist_policy 2y ago
  You just rediscovered that LLMs become much stupider when using images as input. I think that has been already shown for gpt-4 as well.
  - dtquad 2y ago
    Even when using the web search tool GPT-4 becomes stupider. Do they use a dumber model for tool/vision?
    - dwaltrip 2y ago
      I’m guessing that it’s just a much harder problem. Images often contain more information but it is far less structured and refined than language. The transformation process that occurs when people speak and write is incredibly rich and complex. Compared to images which are essentially just the outputs of cameras or screen captures — there isn’t an “intelligent” transformation process occurring.
      - justsid 2y ago
        I think images also have much higher information density than words, or at least they can. There is a reason a picture is worth 1000 words.
Zababa 2y ago
I don't know what's not working but I get "Has a large language model. I don't have the capability to see your screen or any other visual input. My interactions are purely based on the text that you provide"
moffkalast 2y ago
This'll be so fantastic once local models can do it, because nobody in the right mind would stream their voice and everything they do on their machine to Google right? Right? Oh who am I kidding, people upload literally everything to drive lmao.
slowmovintarget 2y ago
At least you can theoretically stop sharing with this one. Microsoft was essentially trying to do this, but doing it for everything on your PC, with zero transparency. Here's Google doing essentially the same thing, even more so that it's explicitly shipping your activity to the cloud, and this response is so different from the "we're sticking this on your machine and you can't turn it off" version Microsoft was attempting to land. This is what Microsoft should have done.
chefandy 2y ago
This is great! I viscerally dislike the "we're going to do art for you so you don't have to... even if you want to..." side of AI, but learning to use the tools to get the satisfaction of making it yourself is not easy! After 2 decades of working with 2D art and code separately, learning 3D stuff (if you include things like the complex and counterintuitive data flow of simulations in Houdini and the like) was as or more difficult than learning to code. Beyond that, taking classes is f'ing expensive, and more of that money goes to educational institutions than the teachers themselves. Obviously, getting beyond the basics for things that require experienced critique are just going to need human understanding, but for the base technical stuff, this is fantastic.
cryptozeus 2y ago
This comment is better than entire ad google just showed. Who is still pointing at the building with camera and asking what is this building?
- kridsdale1 2y ago
  I do that in Manhattan. I also do it for yonder mountains.
icelancer 2y ago
I tried this, shared a terminal, asked it to talk about it, and it guessed that it was Google Chrome with some webUI stuff. Immediately closed the window and bailed.
- kridsdale1 2y ago
  Which terminal? Was it chromium based?
  - icelancer 2y ago
    Nope. Just KiTTY on Windows.

selvan 2y ago

Get started documentation on Multimodal Live API : https://ai.google.dev/api/multimodal-live https://ai.google.dev/api/multimodal-live

Brotkrumen 2y ago

Sounds interesting, but voice input isn't working for me there. I guess I'm too niche with my Mac and Firefox setup.

mentalgear 2y ago
Actually plenty of tech people are using mac & firefox
- portaouflop 2y ago
  Irony detectors malfunctioning perhaps?
  - baq 2y ago
    'irony' meant 'something made of metal' last time I checked
    - socksy 2y ago
      Right, and Macs are made out of aluminium
      - Alifatisk 2y ago
        What is Firefox made out of then?
        shakna 2y ago
        Aluminum. It's American.
        internet_points 2y ago
        The amount of rust indicates iron. So Firefox is very irony.
        cyanwave 2y ago
        Wood.
        dudeinjapan 2y ago
        Fire and foxes, presumably.
    - deleted 2y ago
      [deleted]
littlestymaar 2y ago
This isnt entierly suprising as Google have been breaking things artificially on Firefox for years now (Google Map and YouTube at least). Maybe try spoofing Chrome's user-agent.
SkyPuncher 2y ago
Console is throwing an error: "Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported." Quick research suggests this is part of Firefox's anti-fingerprinting functionality.

sky2224 2y ago

THIS is the thing I'm excited for with AI. I'm someone that becomes about 5x more productive when I have a person watching or just checking in on me (even if they're just hovering there). Having AI to basically be that "parent" to kick me into gear would be so helpful. 90% of the time my problems are because I need someone to help keep the gears turning for me, but there isn't always someone available. This has the potential to be a person that's always available.

Huppie 2y ago
Just as an FYI: I recently learned (here on HN) that this is called Body Doubling[0] there's some services around (there's at least one by someone that hangs around here) that can do this too. [0] https://en.m.wikipedia.org/wiki/Body_doubling https://en.m.wikipedia.org/wiki/Body_doubling
- nolist_policy 2y ago
  Also, there are co-working spaces in VRChat, which works wonders for me. I went to the Glass Office co-working space to study for exams this summer and it worked out really well. I also met some nice people there. A standalone Quest 3 is enough to get you started.
- themaninthedark 2y ago
  Do they support WFH setups?
  - mklepaczewski 2y ago
    The parent might be referring to us: https://workmode.net/ https://workmode.net/. Most of our clients work from home. Do you have a specific concern about body doubling and working from home?
    - kirubakaran 2y ago
      Some unsolicited feedback (please feel free to ignore): When I click on "Pricing" in the nav bar, it scrolls down, and the first thing that catches my eye is "$2100 / month". I happened to see this time that this is the benefit you're projecting and it is actually $2.50/hour. On the previous times I've visited your website based on your HN comments, I've always thought $2100/month was what you were going to charge me and closed the tab. I've been frustrated myself that people don't read what's right there on the page when they come to my startup app's landing page. Turns out I do the same. Hope this helps you improve the layout / font sizes and such "information hierarchy" so the correct information is conveyed at a glance. IMHO $2.50/hour is great value, and stands on its own. I know how much my time is worth, so perhaps the page doesn't really have to shout that to convince me. Again, please feel free to ignore this as it is quite possible that it is just me with the attention span of a goldfish with CTE while clicking around new websites.

internet_points 2y ago

I got so hopefuly by your comment I showed it my current bug that I'm working on, I even prepared everything first with my github issue, the relevant code, the terminal with the failing tests, and I pasted it the full contents of the file and explained carefully what I wanted to achieve. As I was doing this, it repeated back to me everything I said, saying things like "if I understand correctly you're showing me a file called foo dot see see pee" and "I see you have a github issue open called extraneous spaces in frobnicator issue number sixty six" and "I see you have shared some extensive code" and after some more of this "validation"-speak it started repeating the full contents of the file like "backquote backquote backquote forward slash star .. import ess tee dee colon colon .." and so on. Not quite up to excited junior-level programmer standards yet. But maybe good for other things who knows.

nolist_policy 2y ago
You just rediscovered that LLMs become much stupider when using images as input. I think that has been already shown for gpt-4 as well.
- dtquad 2y ago
  Even when using the web search tool GPT-4 becomes stupider. Do they use a dumber model for tool/vision?
  - dwaltrip 2y ago
    I’m guessing that it’s just a much harder problem. Images often contain more information but it is far less structured and refined than language. The transformation process that occurs when people speak and write is incredibly rich and complex. Compared to images which are essentially just the outputs of cameras or screen captures — there isn’t an “intelligent” transformation process occurring.
    - justsid 2y ago
      I think images also have much higher information density than words, or at least they can. There is a reason a picture is worth 1000 words.

Zababa 2y ago

I don't know what's not working but I get "Has a large language model. I don't have the capability to see your screen or any other visual input. My interactions are purely based on the text that you provide"

moffkalast 2y ago

This'll be so fantastic once local models can do it, because nobody in the right mind would stream their voice and everything they do on their machine to Google right? Right? Oh who am I kidding, people upload literally everything to drive lmao.

slowmovintarget 2y ago

At least you can theoretically stop sharing with this one. Microsoft was essentially trying to do this, but doing it for everything on your PC, with zero transparency. Here's Google doing essentially the same thing, even more so that it's explicitly shipping your activity to the cloud, and this response is so different from the "we're sticking this on your machine and you can't turn it off" version Microsoft was attempting to land. This is what Microsoft should have done.

chefandy 2y ago

This is great! I viscerally dislike the "we're going to do art for you so you don't have to... even if you want to..." side of AI, but learning to use the tools to get the satisfaction of making it yourself is not easy! After 2 decades of working with 2D art and code separately, learning 3D stuff (if you include things like the complex and counterintuitive data flow of simulations in Houdini and the like) was as or more difficult than learning to code. Beyond that, taking classes is f'ing expensive, and more of that money goes to educational institutions than the teachers themselves. Obviously, getting beyond the basics for things that require experienced critique are just going to need human understanding, but for the base technical stuff, this is fantastic.

cryptozeus 2y ago

This comment is better than entire ad google just showed. Who is still pointing at the building with camera and asking what is this building?

kridsdale1 2y ago
I do that in Manhattan. I also do it for yonder mountains.

icelancer 2y ago

I tried this, shared a terminal, asked it to talk about it, and it guessed that it was Google Chrome with some webUI stuff. Immediately closed the window and bailed.

kridsdale1 2y ago
Which terminal? Was it chromium based?
- icelancer 2y ago
  Nope. Just KiTTY on Windows.