This was what I had hoped might be possible years ago working on a voice assistant for a browser (Firefox Voice). That was all prototypish and desktop-focused,

This was what I had hoped might be possible years ago working on a voice assistant for a browser (Firefox Voice). That was all prototypish and desktop-focused, but the real potential I saw was in remote control of a cloud-based browser. Voice assistants without a way to execute actions are pretty limited. Google/Siri/Alexa built up those actions through sheer effort, creating integrations for everything, carefully handling the conflicts, etc. A cloud-based browser has the potential to maintain context and perform actions across everything right away, without that huge investment and need for market share, nor buy-in from service providers. I'm not sure how exactly they handle the communication between that browser and the phone, but at least my sense of this is that they are doing what I imagine is "standard" browser automation (which is much more advanced with LLMs) and then something to reinterpret the output for the phone. You can do a lot with the mobile view of the sites, but they seem to be going further. (For something like Spotify I wonder if they are doing something site-specific) The use of a browser offers interesting possibilities for auditing what is happening. With something like GPT Plugins you can see the JSON, but it's not really equivalent to what you as a user understand. Looking at them using Discord, it makes me think they must be doing automation through vision, detecting pixel locations, etc, as Discord is deliberately hard to automate.