Very cool work. Note that this was completed using a vanilla ChatGPT-4o model, all the magic dust is prompting, the dataflow between stages (info gathering, sel

Very cool work. Note that this was completed using a vanilla ChatGPT-4o model, all the magic dust is prompting, the dataflow between stages (info gathering, self-reflection, task inference, skill curation, action planning), and some tooling like added object detection / bounding boxes / icon detection. Also, neither here nor there but I enjoyed the discussion in the paper about how the model had a surprisingly low performance on sending an email in Outlook because while it well-understood the task and how to send an email, Outlook's UI still managed to confuse it - can relate.