Check out the announcement. GPT-4 accepts mixed-mode inputs of text and images. Mouse cursor instructions aren’t a massive leap from the current capabilities,

Check out the announcement. GPT-4 accepts mixed-mode inputs of text and images. Mouse cursor instructions aren’t a massive leap from the current capabilities, given the rate of progress and recent developments around LLM tool use and the like.