3 ms·
Like how LLMs tend to get tripped up by questions that are phrased like well-known riddles, it turns out you can trip up vision models with images that resemble
by jsheard 2y ago
Like how LLMs tend to get tripped up by questions that are phrased like well-known riddles, it turns out you can trip up vision models with images that resemble well-known optical illusions. Overfitting strikes again.
https://x.com/kosa12matyas/status/1871256745403953572 https://x.com/kosa12matyas/status/1871256745403953572
- Jackson__ 2y agoIn essence, that is what HallusionBench[0] does. While I think it is an improvement over other vision benchmarks, it still falls short in terms of quantifying actual vision capabilities. More than anything, it seems like a way to detect whether the model was over trained on these riddles. [0] https://github.com/tianyi-lab/HallusionBench https://github.com/tianyi-lab/HallusionBench