abrichr

10 ms·

91.
by abrichr 2y ago
Thanks! OpenAdapt already supports audio recording during demonstration ( https://github.com/OpenAdaptAI/OpenAdapt/pull/346 ). Perhaps I misunderstood — can you please clarify your suggestion?
92.
by abrichr 2y ago
Very interesting, thank you for making this available! At OpenAdapt ( https://github.com/OpenAdaptAI/OpenAdapt ) we are looking into using pm4py ( https://github.com/pm4py ) to extract a process graph from
93.
by abrichr 2y ago
Thank you for making this available! Check out https://github.com/OpenAdaptAI/OpenAdapt for a cross platform (Mac and Windows) open source library that learns to perform tasks in desktop apps by observing human demonst
94.
by abrichr 2y ago
https://ifaketextmessage.com/K7dQ/
95.
by abrichr 2y ago
https://github.com/OpenAdaptAI/OpenAdapt runs locally on your machine to automate repetitive tasks in desktop apps. We have plans to build a mobile interface as well.
96.
by abrichr 2y ago
See https://github.com/OpenAdaptAI/OpenAdapt for an alternative that works with desktop GUIs.
97.
by abrichr 2y ago
For an open source alternative, check out https://github.com/OpenAdaptAI/OpenAdapt . We combine Segment Anything Model with GPT4-V to understand recordings of workflows in desktop user interfaces, then replay them accor
98.
by abrichr 2y ago
Not mentioned: learning from demonstration. This is the approach we are taking at https://github.com/OpenAdaptAI/OpenAdapt .
99.
by abrichr 3y ago
Perhaps de-identification before training could be helpful here. Microsoft does seem active in this, e.g. https://microsoft.github.io/presidio/
100.
by abrichr 3y ago
Update: it looks like Anthropic now accepts Canadian credit cards. The results are not as good as GPT4-V or Gemini. I've posted the output for each of `gpt-4-vision-preview`, `gpt-4-turbo-2024-04-09`, `gemini-1.5-pro-latest`, and `clau
101.
by abrichr 3y ago
SAM is used to segment GUI elements, which are then sent to GPT4-V to be described. We then prompt GPT-4 with the user's actions and the descriptions of the GUI elements to generate new actions to accomplish different behaviors.
102.
by abrichr 3y ago
You can see how it performs in describing GUI elements in the linked PR if you scroll down -- here's a direct link: https://private-user-images.githubusercontent.com/774615/320... Regarding failure modes, we have
103.
by abrichr 3y ago
Thank you!
104.
by abrichr 3y ago
We've been using `gpt-4-1106-vision-preview` and simply prompting the model to return json, with excellent results: https://github.com/OpenAdaptAI/OpenAdapt/pull/610 (work in progress).
105.
by abrichr 3y ago
We haven't been able to use Claude 3 Opus vision yet because we're in Canada, but GPT-4-V works extremely well (when combined with Segment Anything). See: https://github.com/OpenAdaptAI/OpenAdapt/pull&#x2
106.
by abrichr 3y ago
At OpenAdapt we have had excellent results combining Segment Anything Model (SAM) with GPT-4 for screen understanding. Work-in-progress: https://github.com/OpenAdaptAI/OpenAdapt/pull/610
107.
by abrichr 3y ago
https://github.com/OpenAdaptAI/OpenAdapt is designed for all desktop applications, including browsers. We're also working on a chrome extension to support reading/writing directly to DOM: https://g
108.
by abrichr 3y ago
From https://github.com/gptscript-ai/gptscript/blob/main/docs/REA... : > In GPTScript, agents and assistants are implemented using tools. Tools can use other tools. Tools can be implemented using
109.
by abrichr 3y ago
Congratulations on shipping! Check out https://github.com/OpenAdaptAI/OpenAdapt for an open source (MIT license) alternative that also works on desktop (including Citrix!)
110.
by abrichr 3y ago
Or let https://github.com/OpenAdaptAI/OpenAdapt do the boring tasks for you!
111.
by abrichr 3y ago
Thanks! Are there any drawbacks to Pywebview compared to a browser?
112.
by abrichr 3y ago
What are the advantages of using something like Pywebview compared to opening the user’s browser to localhost?
113.
by abrichr 3y ago
Congratulations on shipping! You might find this interesting, a comment on https://news.ycombinator.com/item?id=35376138 ( https://news.ycombinator.com/item?id=35379983 ): > I had been playing with the ide
114.
by abrichr 3y ago
SEEKING FREELANCER | Remote OpenAdapt.AI is pioneering open source AI-First Process Automation by connecting Large Multimodal Models with GUIs for efficient automation. Our open-source initiative aims to eliminate tedious manual tasks, maki
115.
by abrichr 3y ago
OpenAdapt.AI | Remote | Contract At OpenAdapt.AI, we're at the forefront of open source AI-First Process Automation, leveraging Large Multimodal Models (LMMs) to revolutionize how we interact with traditional desktop and web Graphical
116.
by abrichr 3y ago
We are building this at https://openadapt.ai , except the user specifies when to record.
117.
by abrichr 3y ago
To add to this, from https://en.wikipedia.org/wiki/List_of_countries_by_motor_veh... : # Country 2022 ---------------------------- 1 China 27,020,615 2 United States 10,060,339
118.
by abrichr 3y ago
At https://openadapt.ai/ we are using LLMs to automate repetitive tasks in GUI interfaces. Think robotic process automation, but via learning from demonstration rather than no-code scripting. The stack is mostly python runn
119.
by abrichr 3y ago
SEEKING VOLUNTEERS: OpenAdapt.AI: AI-First Process Automation OpenAdapt is the world's first open source AI-first process automation framework. If you're interested in the promise of Large Language Models (LLMs) and Large Multimod
120.
by abrichr 3y ago
Carmack has partnered with Rich Sutton, who is an AI researcher -- and, according to https://www.amii.ca/latest-from-amii/john-carmack-and-rich-s... : > the principal founder of the field of reinforcement learning.

More ›