Interesting, I wonder if there's a similar approach that can be taken for complex tasks in the "multimodal" virtual environment of a computer desktop or phone,

Interesting, I wonder if there's a similar approach that can be taken for complex tasks in the "multimodal" virtual environment of a computer desktop or phone, working across/between multiple applications and web pages with various inputs and interactions. Taking a more general approach that might be closer to how humans work then ACT-1 and the like.