5 ms·
> With onscreen awareness, Siri will be able to understand and take action with users’ content in more apps over time. For example, if a friend texts a user the
by abrichr 2y ago
> With onscreen awareness, Siri will be able to understand and take action with users’ content in more apps over time. For example, if a friend texts a user their new address in Messages, the receiver can say, “Add this address to his contact card.”
I wonder how they will extend this to business processes that are not in their training set. At https://openadapt.ai https://openadapt.ai we rely on users to demonstrate tasks, then have the model analyze these demonostrations in order to automate them.
- ducadveritatem 2y agoCan’t find a link right now (search terms are pretty crowded atm!) but I saw they just recently shared an llm they’ve been working on that is designed to answer questions about how a give screen of an app functions. (Identifying buttons, core functionality, etc.)