3 ms·

Could someone with access tell me what GPT-4V has to say of this image? http://karpathy.github.io/assets/obamafunny.jpg http://karpathy.github.io/assets/obamaf

by fdeage 3y ago

Could someone with access tell me what GPT-4V has to say of this image? http://karpathy.github.io/assets/obamafunny.jpg http://karpathy.github.io/assets/obamafunny.jpg Andrej Karpathy used it in 2012 as an example of an image he thought would be extremely hard for a model to interpret. I'm wondering how this holds 11 years later...

deleted 3y ago
[deleted]
zamadatix 3y ago
Prompt: "What do you have to say about this image?" Response 1: "The image depicts a candid moment in what appears to be a hallway or corridor setting. Several individuals are present, engaging in various activities. The prominent features include a man on the left, who seems to be peering into a room, and another man next to him standing on a weighing scale and taking some notes. The individuals in the background are conversing with each other. The overall mood seems light-hearted and jovial. The architecture and interior suggest an institutional or official setting, possibly an office building or government facility. The tiled floor, lockers, and signs on the walls contribute to this formal ambiance. Overall, it's a snapshot of a seemingly casual moment in a professional setting." Response 2: "Sorry, I cannot help with that." New chat, Prompt: "Why is this image funny?" Response 1: "The humor in this image seems to stem from the juxtaposition of formal and informal elements. Here are a few reasons why someone might find it amusing: Unexpected Situation: Important figures, who are usually seen in serious or formal settings, are captured in a more relaxed, candid, and human moment. Height Discrepancy: The taller individual is using a device to measure height, while the shorter individual walks by. This could create a comedic contrast between their heights. Expressions: The reactions of people in the background and the focal subjects can be humorous. They are smiling or laughing, which indicates something amusing might have occurred or been said. Informal Setting: The backdrop of lockers and checkered floor tiles gives the feel of a school or gym environment, which contrasts with the formal attire of the individuals. Different people might find humor in different elements of the photo. It's a mix of candidness, unexpected situations, and visual contrasts that create a light-hearted and humorous scene." Response 2: (same) New chat, prompt: "What is the person in the middle doing with their foot and why?" Response: "The person in the middle appears to be playfully stepping on the scale while the taller individual is weighing himself, likely as a humorous gesture to momentarily increase the weight reading. It's a light-hearted way of interacting and bringing some humor to the situation." . Apologies for how bad the formatting of this is going to come out, not sure how to make it better on HN (wish we had real quotes not just code blocks). Overall, I don't think it either noticed the foot was on the scale by itself or put it together that this was the focus until fed that information. Otherwise it was more lost in generalities about the image.
- derangedHorse 3y ago
  Still cool that it elaborated once you pointed it out
- fdeage 3y ago
  Thank you. The quality of the answer it gave you is on par with what I was expecting, except I am very surprised it didn't recognize Obama. Also, do you know why this response differ so much from ( and is far less accurate than) this response? https://news.ycombinator.com/item?id=37674968 https://news.ycombinator.com/item?id=37674968 In the previous link, GPT4V seems to have understood everything there was to understand about the picture (suspiciously, btw. As someone else said it, the picture and its text are almost certainly in the training data).
  - zamadatix 3y ago
    I think it comes down to what you specifically put in the prompt. If you frame your question around the people specifically then ChatGPT will frame its response around as much as it knows or can invent about the people as well. It doesn't really have a good concept itself of what to focus on talking about outside of that, hence the lame generalizations around the very generic prompt I started with.
ediardo 3y ago
Bard: Prompt: What's funny about this image? Bard: Sorry, I can't help with images of people yet.

by fdeage 3y ago

deleted 3y ago

[deleted]

zamadatix 3y ago

Prompt: "What do you have to say about this image?" Response 1: "The image depicts a candid moment in what appears to be a hallway or corridor setting. Several individuals are present, engaging in various activities. The prominent features include a man on the left, who seems to be peering into a room, and another man next to him standing on a weighing scale and taking some notes. The individuals in the background are conversing with each other. The overall mood seems light-hearted and jovial. The architecture and interior suggest an institutional or official setting, possibly an office building or government facility. The tiled floor, lockers, and signs on the walls contribute to this formal ambiance. Overall, it's a snapshot of a seemingly casual moment in a professional setting." Response 2: "Sorry, I cannot help with that." New chat, Prompt: "Why is this image funny?" Response 1: "The humor in this image seems to stem from the juxtaposition of formal and informal elements. Here are a few reasons why someone might find it amusing: Unexpected Situation: Important figures, who are usually seen in serious or formal settings, are captured in a more relaxed, candid, and human moment. Height Discrepancy: The taller individual is using a device to measure height, while the shorter individual walks by. This could create a comedic contrast between their heights. Expressions: The reactions of people in the background and the focal subjects can be humorous. They are smiling or laughing, which indicates something amusing might have occurred or been said. Informal Setting: The backdrop of lockers and checkered floor tiles gives the feel of a school or gym environment, which contrasts with the formal attire of the individuals. Different people might find humor in different elements of the photo. It's a mix of candidness, unexpected situations, and visual contrasts that create a light-hearted and humorous scene." Response 2: (same) New chat, prompt: "What is the person in the middle doing with their foot and why?" Response: "The person in the middle appears to be playfully stepping on the scale while the taller individual is weighing himself, likely as a humorous gesture to momentarily increase the weight reading. It's a light-hearted way of interacting and bringing some humor to the situation." . Apologies for how bad the formatting of this is going to come out, not sure how to make it better on HN (wish we had real quotes not just code blocks). Overall, I don't think it either noticed the foot was on the scale by itself or put it together that this was the focus until fed that information. Otherwise it was more lost in generalities about the image.

derangedHorse 3y ago
Still cool that it elaborated once you pointed it out
fdeage 3y ago
Thank you. The quality of the answer it gave you is on par with what I was expecting, except I am very surprised it didn't recognize Obama. Also, do you know why this response differ so much from ( and is far less accurate than) this response? https://news.ycombinator.com/item?id=37674968 https://news.ycombinator.com/item?id=37674968 In the previous link, GPT4V seems to have understood everything there was to understand about the picture (suspiciously, btw. As someone else said it, the picture and its text are almost certainly in the training data).
- zamadatix 3y ago
  I think it comes down to what you specifically put in the prompt. If you frame your question around the people specifically then ChatGPT will frame its response around as much as it knows or can invent about the people as well. It doesn't really have a good concept itself of what to focus on talking about outside of that, hence the lame generalizations around the very generic prompt I started with.

ediardo 3y ago

Bard: Prompt: What's funny about this image? Bard: Sorry, I can't help with images of people yet.