3 ms·
Or they simply don’t have that information. OpenAI models have done badly traditionally on placement because the encoding of the image doesn’t include the infor
by IanCal 1y ago
Or they simply don’t have that information. OpenAI models have done badly traditionally on placement because the encoding of the image doesn’t include the information very well. Gemini is better as it seems to be passed pre segmented images with bounding box info.
It’s similar to the counting letters problem - they’re not seeing the same thing you are .
On a simple practical level it’s irrelevant whether your problem is not solved because the model can’t understand or the image encoding is useless. However to understand what the models could be capable of it’s a poor test. Like asking how well I can play chess then saying I’m bad at it after watching me play by feel in thick gloves.
- suddenlybananas 1y agoHow does that apply in any way to this example?
- foobarbecue 1y agoI think it applies. Presumably training data is enough to put humans in the front seats in a car, but lacks info on which seat is the driver's seat, or which person was the driver. Maybe I should have tried "steering wheel".
- IanCal 1y agoImagine being asked to draw what the op said, but you couldn’t see what you’d drawn - only a description that said “a man and a woman in a Honda “ Asked to draw a new picture with the history of : Draw a picture of a man in the driver seat and a woman in the passenger seat. (Picture of a man and a woman in a car) No, the man in the drivers seat! —— How well do you think a very intelligent model could draw the next picture? It failed the first time and the descriptions mean it has no idea what it even drew before.
- charcircuit 1y agoCoding agents have had good success doing this. Providing the errors allows it to potentially figure out how to fix it. It's able to do more with this iterative approach than without.
- IanCal 1y agoBut fundamentally it requires that it can actually see the thing it’s trying to fix. Lots of these models can essentially barely see.