2 ms·
Perception is just 1-2 screenshots. A number of recent VLM models have a lot more pretraining data on GUI interactions, which helps.
by mountainriver 2y ago
Perception is just 1-2 screenshots. A number of recent VLM models have a lot more pretraining data on GUI interactions, which helps.
- iiJDSii 2y agoSuch as? Are they able to recognize arbitrary GUI elements from various desktop programs, web browsers, etc?
- mountainriver 2y agoQwen2.5-vl seems to be the best right now by our tests. UI-TARS by bytedance also has a good amount of pretraining. Molmo is also very good at coordinates.