3 ms·
No modern LLM can tell how many eyes the magic card Pit Imp has. They all say 2. This is across all reasoning levels and paid Gemini, Claude, and GPT (Sol) D
by piyh 19d ago
No modern LLM can tell how many eyes the magic card Pit Imp has. They all say 2. This is across all reasoning levels and paid Gemini, Claude, and GPT (Sol)
Drawing a line red to split up the image then has them answer correctly.
Their failure modes are highly correlated.