A while ago I tested the image recognition skills of GPT-4o, Claude 3, and Gemini using a random street plan diagram I happened to have in my downloads folder (

A while ago I tested the image recognition skills of GPT-4o, Claude 3, and Gemini using a random street plan diagram I happened to have in my downloads folder (https://i.imgur.com/9WZpK0L.png https://i.imgur.com/9WZpK0L.png). It's a top-down CAD rendering showing the planned renovation of a street in my neighborhood in Tampere, Finland. I uploaded the image and simply asked each model "What can you tell me about the attached picture?" GPT-4o's answer was excellent and highly detailed, recognizing essentially all the relevant aspects of the image [GPT4o]. Claude 3 Sonnet was correct on a general level, but its answer was much less detailed and showed more uncertainty in the form of "A or B" sentences [CL3]. Gemini's answer was, well, hilariously wrong [GEM]. I just tried this with Claude 3.5 Sonnet and it did very well. Its answer was still not as detailed as GPT-4o's, but it did ask me if I want it to elaborate on any aspect of the image [CL35]. I think this was an interesting experiment because street plan CAD diagrams probably aren't very common in the training data of these models. -- [GPT4o] https://gist.github.com/jdahlstrom/844bda8ac76a5c3248c863d20ac79128 https://gist.github.com/jdahlstrom/844bda8ac76a5c3248c863d20... [CL3] https://gist.github.com/jdahlstrom/ecccf31c8305f82519f27af5312f6656 https://gist.github.com/jdahlstrom/ecccf31c8305f82519f27af53... [GEM] https://gist.github.com/jdahlstrom/2e12a966c0d603a7b1446ba085ec927b https://gist.github.com/jdahlstrom/2e12a966c0d603a7b1446ba08... [CL35] https://gist.github.com/jdahlstrom/60ca9352630934bec6e2f4e37062fbbd https://gist.github.com/jdahlstrom/60ca9352630934bec6e2f4e37...