3 ms·

Wiggling the mouse is what people do involuntarily when the computer isn’t working right. They are setting themselves up for Gemini to be the uninvited Clippy,

Wiggling the mouse is what people do involuntarily when the computer isn’t working right. They are setting themselves up for Gemini to be the uninvited Clippy, except this will send everything you are working on to Google to harvest data from.

paxys 5mo ago
It is deliberately designed for maximum accidental invocations so the managers and execs behind it can claim the large user numbers in their promo packets.
jeroenhd 5mo ago
The video they show (which is probably exaggerated by cutting out LLM generation time) is pretty sci-fi. I don't know how it works in practice, but it looks fun to try out. If this could run locally, I'd love to have a feature like that. Most people don't really seem to care about data collection when it comes to AI usage. A lot of people who will feed Gemini/ChatGPT/Bing/Claude/shady clusters across the internet for bargain bin prices/Mistral every detail of their lives will probably be fine with Gemini as long as it doesn't interfere unnecessarily.
- bel8 5mo ago
  It probably works similar to how Gemini works in Android for a while now. You can point or select anywhere on the screen and it understands and searches the context. If you select a text block, even text inside an image, it allows to copy or search the text online. Otherwise it can search the image. I use it often. It's intuitive and fast even on non-flagship phones. I'd wager their A/B tests went well enough to warrant a port from phones to their new "Chromebook".
  - jeroenhd 5mo ago
    Their video is completely different from what Gemini does now. It analyses mouse movements, like circling around things, underlining things with the mouse, pointing at things to indicate where they need to go. It's a lot like the interfaces you might see in sci-fi movies, where generic gestures are understood within context in a way that modern computers can't handle.
    - bel8 5mo ago
      > circling around things, underlining things with the mouse Do we use the same Android Gemini assistant? Because the one I use does that and it has object detection smart enough to be intuitive. It usually gets it right when I point something on the screen. And when it doesn't, I can circle around the thing or just click again. This Instagram post for example, it automatically highlighted the entire person, but I wanted to know about the shoes. I then clicked once on the shoes and it knew exactly what I wanted and gave me the info in about 2 seconds: https://imgur.com/a/lHUeciy https://imgur.com/a/lHUeciy This is useful to non tech savvy folks. Not just to us hackers.
      - jeroenhd 5mo ago
        Google's Gemini features differ per region to a massive extent. There's a good chance privacy laws prevent Google from providing me with the same Gemini you use. Object detection is mediocre at best. Circling things and using their AI editing features works, but the artefacts confuse Lens and other image parsing systems. Extracting objects from images usually mostly works, but it's not on par with what Apple had long before Google built it. The difference remains that the Gemini app on Android requires activation. You cannot tap a button or click a link while you're on the Gemini screen. The video isn't on the linked page anymore, but it's here: https://deepmind.google/blog/ai-pointer/ https://deepmind.google/blog/ai-pointer/ and here: https://www.youtube.com/watch?v=pZNzfQLgGsA https://www.youtube.com/watch?v=pZNzfQLgGsA It's an absolute privacy nightmare for most people, but if we ever get enough RAM and compute to run this stuff locally, I think this can actually make a new paradigm for user interaction, something with lisp machine self-customisability but for people who don't know anything about computers. And if it doesn't work, it'll be the most horrific, messy, useless UI humanity has ever invented, and we all get a new funny meme to laugh about Google. Win-win!
- deckar01 5mo ago
  > Most people don't really seem to care about data collection when it comes to AI usage. That assumes you intended to use AI. People are going to accidentally upload random private content to google.
  - jeroenhd 5mo ago
    If you buy the Google Gemini AI Agentic Laptop or whatever they will market this as, you're going to want to try AI. What else is the point of buying a Chromebook, as nice and slick as it may look, when similar or even better alternatives exist.
dmonitor 5mo ago
It's the unofficial "where's my mouse pointer" macro At least one DE I've used (MacOS? KDE?) even had it as an official macro that would make the pointer 10x bigger when you shook it
- scblock 5mo ago
  KDE does that by default. Handy sometimes, funny sometimes.
  - mkl 5mo ago
    If you keep on shaking the mouse, the pointer just keeps getting bigger and bigger. You can certainly find it when it is enormous.
- crabmusket 5mo ago
  MacOS and KDE both do this. In KDE the pointer keeps getting larger the longer you shake it until it is truly absurdly huge.
  - cevn 5mo ago
    This might be the new Compiz, my friends love seeing this effect on KDE.

paxys 5mo ago

It is deliberately designed for maximum accidental invocations so the managers and execs behind it can claim the large user numbers in their promo packets.

jeroenhd 5mo ago

The video they show (which is probably exaggerated by cutting out LLM generation time) is pretty sci-fi. I don't know how it works in practice, but it looks fun to try out. If this could run locally, I'd love to have a feature like that. Most people don't really seem to care about data collection when it comes to AI usage. A lot of people who will feed Gemini/ChatGPT/Bing/Claude/shady clusters across the internet for bargain bin prices/Mistral every detail of their lives will probably be fine with Gemini as long as it doesn't interfere unnecessarily.

bel8 5mo ago
It probably works similar to how Gemini works in Android for a while now. You can point or select anywhere on the screen and it understands and searches the context. If you select a text block, even text inside an image, it allows to copy or search the text online. Otherwise it can search the image. I use it often. It's intuitive and fast even on non-flagship phones. I'd wager their A/B tests went well enough to warrant a port from phones to their new "Chromebook".
- jeroenhd 5mo ago
  Their video is completely different from what Gemini does now. It analyses mouse movements, like circling around things, underlining things with the mouse, pointing at things to indicate where they need to go. It's a lot like the interfaces you might see in sci-fi movies, where generic gestures are understood within context in a way that modern computers can't handle.
  - bel8 5mo ago
    > circling around things, underlining things with the mouse Do we use the same Android Gemini assistant? Because the one I use does that and it has object detection smart enough to be intuitive. It usually gets it right when I point something on the screen. And when it doesn't, I can circle around the thing or just click again. This Instagram post for example, it automatically highlighted the entire person, but I wanted to know about the shoes. I then clicked once on the shoes and it knew exactly what I wanted and gave me the info in about 2 seconds: https://imgur.com/a/lHUeciy https://imgur.com/a/lHUeciy This is useful to non tech savvy folks. Not just to us hackers.
    - jeroenhd 5mo ago
      Google's Gemini features differ per region to a massive extent. There's a good chance privacy laws prevent Google from providing me with the same Gemini you use. Object detection is mediocre at best. Circling things and using their AI editing features works, but the artefacts confuse Lens and other image parsing systems. Extracting objects from images usually mostly works, but it's not on par with what Apple had long before Google built it. The difference remains that the Gemini app on Android requires activation. You cannot tap a button or click a link while you're on the Gemini screen. The video isn't on the linked page anymore, but it's here: https://deepmind.google/blog/ai-pointer/ https://deepmind.google/blog/ai-pointer/ and here: https://www.youtube.com/watch?v=pZNzfQLgGsA https://www.youtube.com/watch?v=pZNzfQLgGsA It's an absolute privacy nightmare for most people, but if we ever get enough RAM and compute to run this stuff locally, I think this can actually make a new paradigm for user interaction, something with lisp machine self-customisability but for people who don't know anything about computers. And if it doesn't work, it'll be the most horrific, messy, useless UI humanity has ever invented, and we all get a new funny meme to laugh about Google. Win-win!
deckar01 5mo ago
> Most people don't really seem to care about data collection when it comes to AI usage. That assumes you intended to use AI. People are going to accidentally upload random private content to google.
- jeroenhd 5mo ago
  If you buy the Google Gemini AI Agentic Laptop or whatever they will market this as, you're going to want to try AI. What else is the point of buying a Chromebook, as nice and slick as it may look, when similar or even better alternatives exist.

dmonitor 5mo ago

It's the unofficial "where's my mouse pointer" macro At least one DE I've used (MacOS? KDE?) even had it as an official macro that would make the pointer 10x bigger when you shook it

scblock 5mo ago
KDE does that by default. Handy sometimes, funny sometimes.
- mkl 5mo ago
  If you keep on shaking the mouse, the pointer just keeps getting bigger and bigger. You can certainly find it when it is enormous.
crabmusket 5mo ago
MacOS and KDE both do this. In KDE the pointer keeps getting larger the longer you shake it until it is truly absurdly huge.
- cevn 5mo ago
  This might be the new Compiz, my friends love seeing this effect on KDE.