38 ms·
OmniParser V2 – A simple screen parsing tool towards pure vision based GUI agent
- icodar 2y agoThis is not the intended use but it good working on parsing document layout from image.
- NewUser76312 2y agoVery cool work. Accurate GUI text and element parsing is exactly the kind of input that LLMs need to be effective agents.
- rgovostes 2y agoThe OS has additional information including how different graphics layers are composited, and what accessibility metadata is attached to interface elements. It ought to be useful to exploit this to do better than screenshot parsing.
- nighthawk454 2y agoOne ponders the connections with the Recall feature