3 ms·

Congratulations on shipping! In https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt/strategies/visual.py https://github.com/OpenAdaptAI/OpenAdapt/blob

Congratulations on shipping! In https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt/strategies/visual.py https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt... we use FastSAM to first segment the UI elements, then have the LLM describe each segment individually. This seems to work quite well; see https://twitter.com/OpenAdaptAI/status/1789430587314336212 https://twitter.com/OpenAdaptAI/status/1789430587314336212 for a demo. More coming soon!

jackienotchan 2y ago
Looking at OpenAdapt, I'm wondering why they didn't integrate Tarsier into AgentGPT, which is their flagship github repo but doesn't seem to be under active development anymore.
- KhoomeiK 2y ago
  We have a lot more powerful use-cases for Tarsier in web data extraction at the moment. Stay tuned for a broader launch soon!